InfoBFR: Real-World Blind Face Restoration via Information Bottleneck

Nan Gao, Jia Li, Huaibo Huang, Zhi Zeng, Ran He

Introduction

Blind Face Restoration (BFR) is the challenging task of recovering high-quality (HQ) facial images from low-quality (LQ) inputs plagued by unknown blurring, noise, or compression. While recent advancements in Generative Adversarial Networks (GANs) and diffusion-based models—such as CodeFormer, GPEN, and DiffBIR—have significantly improved restoration quality, they often struggle in complex, real-world "out-of-domain" scenarios.

InfoBFR introduces a novel, plug-and-play framework designed to address a specific phenomenon the authors call neural degradations. These are not artifacts from the original camera but are instead errors, hallucinations, and distortions introduced by the restoration models themselves during the inference process.

The Challenge: Neural Degradations

Even state-of-the-art (SOTA) models frequently introduce visual glitches when attempting to reconstruct a face. InfoBFR categorizes these neural degradations into three primary types:

  • Prior Bias: The model forces a face to resemble a generic training sample rather than the specific individual in the source image.
  • Topological and Textural Distortion: The creation of unnatural eye shapes or "plastic," overly smooth skin textures.
  • Artifact Residues: The presence of visual hallucinations or strange glitches that do not exist in the original low-quality input.

While traditional methods focus on removing data degradation (like sensor noise), InfoBFR acts as a "restorer for restorers," polishing the outputs of existing models to eliminate the noise they inadvertently generate.

The InfoBFR Framework

InfoBFR is designed as a universal enhancement layer that processes the output of a pre-trained BFR model through two distinct stages: Information Compression and Information Compensation.

1. Information Compression: The Manifold Information Bottleneck (MIB)

Inspired by information theory, the researchers utilize a Manifold Information Bottleneck (MIB) to "absorb the essence and discard the useless." This stage uses a Transformer-based attention mechanism to identify which features from the base model are reliable and which are likely neural degradations. By minimizing the mutual information between the distorted input and the optimized manifold, the MIB effectively filters out artifacts while preserving the underlying facial structure.

2. Information Compensation: One-Step Diffusion Refinement

Filtering out artifacts can sometimes leave the image lacking in fine-grained detail. To restore high-fidelity features—such as teeth, skin pores, and hair—InfoBFR employs Stable Diffusion (v2.1). To maintain computational efficiency, the authors use Low-Rank Adaptation (LoRA) to fine-tune the model into a one-step generator. This allows the system to synthesize realistic details that conform to real-world face distributions without the heavy latency typically associated with standard diffusion processes.

An adaptive information filter ((\lambda)) is also introduced, allowing the system to dynamically decide how much information to retain from the base model versus how much to allow the diffusion model to "re-imagine."

Experimental Results and Insights

InfoBFR was trained on the FFHQ (Flickr-Faces-HQ) dataset and evaluated across four rigorous real-world datasets: Wider-Test (severe degradation), CelebChild (color-shifted/blurry photos), WebPhoto (internet-sourced LQ faces), and FOS (side-view or occluded faces).

Key Findings:

  • Universal Boosting: InfoBFR successfully improved the performance of 10 different SOTA models, demonstrating its versatility as a plug-and-play module.
  • Quantitative Dominance: In head-to-head evaluations across 16 events, InfoBFR-equipped models secured 10 first prizes and 12 second prizes in critical metrics like FID (image quality) and NIQE (naturalness).
  • Qualitative Superiority: Visual analysis confirms that InfoBFR effectively repairs "melted" facial features and removes hallucinated textures, particularly in cases involving heavy occlusion (e.g., faces behind microphones or hands).
  • Efficiency: Despite its diffusion-based backend, InfoBFR remains highly efficient. It utilizes only 16M trainable parameters and achieves an inference time of approximately 70ms on an RTX 4090, making it viable for practical applications.

Limitations and Future Directions

While InfoBFR is highly robust, the authors note that in extreme scenarios involving near-total data loss or very high occlusion, the model can still produce slight artifacts. Additionally, while 70ms is fast for diffusion, it remains slower than pure GAN-based methods that operate in the 10-20ms range.

Future work may focus on even more efficient information optimization strategies to handle diverse, non-human facial data and further reduce the computational overhead, moving closer to a truly universal, real-time restoration standard.

Summary updated

Abstract

Previous blind face restoration (BFR) methods have primarily leveraged facial priors from pretrained GAN or diffusion models. These neural BFR models suffer from diverse neural degradations, such as prior bias, topological distortion, textural distortion, and artifact residues, which limit their real-world generalization in complex real-world scenarios. In this paper, we propose an effective framework, <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">InfoBFR</i>, to address neural degradation from an information-theoretic perspective, which achieves BFR boosting in diverse wild and heterogeneous scenes. Specifically, on the basis of the results from pretrained BFR models, InfoBFR considers information compression by using a manifold information bottleneck (MIB) and manifold information compensation (MIC) with efficient diffusion LoRA to conduct information optimization. InfoBFR effectively synthesizes high-fidelity faces with texture and structure boosting. Comprehensive experimental results demonstrate the high boosting performance of InfoBFR (nearly 82%) for state-of-the-art GAN-based and diffusion-based BFR methods, as it can complete BFR tasks in approximately 70 ms and has 4M trainable parameters. It is promising that InfoBFR is the first unified postprocessing restorer universally employed by diverse BFR models to overcome the limitations of neural degradation.
BibTeX
@article{gao2025infobfr,
  author = {Nan Gao and Jia Li and Huaibo Huang and Zhi Zeng and Ran He},
  title = {{InfoBFR: Real-World Blind Face Restoration via Information Bottleneck}},
  journal = {IEEE Transactions on Circuits and Systems for Video Technology},
  year = {2025},
  doi = {10.1109/tcsvt.2025.3637231},
}
Figure