Introduction
Blind Face Restoration (BFR) aims to recover high-quality facial images from inputs suffering from unknown, complex degradations such as blur, noise, and compression. While modern GAN and Transformer-based models produce sharp results, they often suffer from two major flaws: low generalization to non-photorealistic domains (e.g., paintings, statues) and the generation of "hallucination" artifacts—incorrect details like double eyelashes or distorted textures that destroy the subject's identity.
The DiffMAC Framework
To address these challenges, the authors propose DiffMAC, a two-stage Diffusion-Information-Diffusion (DID) framework. The core innovation is the Manifold Information Bottleneck (MIB), which acts as a sophisticated filter between two diffusion stages to ensure both quality and fidelity.
Two-Stage Restoration Process
- Stage I (Alignment): A latent diffusion model aligns the restored face with the spatial features of the low-quality (LQ) input using Adaptive Instance Normalization (AdaIN). This removes primary degradations but can still produce uncontrollable artifacts.
- Manifold Information Bottleneck (MIB): This is the framework's central mechanism. It uses information theory to compress the image "manifold" (the mathematical representation of the face), maximizing the preservation of structural identity while minimizing the influence of noise and restoration-irrelevant details. This prevents the model from hallucinating features that weren't in the original.
- Stage II (Refinement): The model uses the compressed manifold and identity injection (via ArcFace embeddings) to fine-tune the results. This ensures the output is high-resolution and faithful to the original person's identity and the specific artistic style of the scene.
Experimental Results
The model was trained on the standard FFHQ dataset and evaluated across diverse benchmarks, including synthetic (CelebA-HQ), real-world (LFW-Test, CelebChild), and a new Heterogeneous Face in the Wild (HFW) dataset containing 1,200 images of sculptures, oil paintings, and anime characters.
Key Performance Metrics
- Quantitative Superiority: On the LFW-Test dataset, DiffMAC achieved a state-of-the-art FID score of 37.75, significantly outperforming GAN-based methods like CodeFormer (52.02) and other diffusion models like DiffBIR (38.97).
- Heterogeneous Generalization: In subjective user studies on the HFW dataset, DiffMAC scored 4.4 for style preservation, nearly doubling the performance of CodeFormer (2.5). While other models often add "human" eyes to statues or messy textures to paintings, DiffMAC maintains the original medium's texture while sharpening facial features.
- Identity Preservation: By filtering noise through the MIB, the model avoids common generative artifacts, ensuring that the restored face remains recognizable as the original subject.
Limitations and Future Work
Despite its performance, DiffMAC's two-stage diffusion process is computationally expensive. It takes approximately 10.78 seconds to process a single image on an NVIDIA RTX 3090, which is considerably slower than GAN-based alternatives. Additionally, the model can still struggle with extremely degraded contours, such as ambiguous jawlines. Future research will explore efficient distillation techniques to accelerate inference for real-time applications.
Summary updated