Introduction
Blind face restoration (BFR) from extremely low-quality (LQ) images is a challenging "one-to-many" inverse problem. A single degraded input can correspond to numerous plausible high-resolution (HR) faces, varying in identity, accessories, or fine details. Traditional methods often produce a single deterministic output, failing to capture this inherent uncertainty.
This paper introduces Measurement-Constrained Sampling (MCS), a training-free approach that leverages pre-trained text-to-image (T2I) diffusion models. By using text prompts, MCS allows users to navigate the solution space, generating diverse and high-quality reconstructions that are both structurally consistent with the input and semantically aligned with user descriptions.
Methodology
MCS is a zero-shot method that modifies the sampling trajectory of a pre-trained diffusion model (e.g., Deep-Floyd IF). It employs a staged Selection Mechanism to balance structural fidelity with generative diversity:
- Forward Measurement (Early Phase): During the initial denoising steps, the model focuses on global structure. MCS uses Haar wavelet decomposition to align the generated image's low-frequency components with a coarse HR estimate from a pre-trained restorer. This prevents structural hallucinations.
- Reverse Measurement (Later Phase): As the image stabilizes, the constraint shifts to a pseudo-inverse operator. This defines a "solution space" of valid images that would degrade into the original LQ input.
- Text-Guided Exploration: Within this relaxed solution space, text prompts (e.g., "a woman with glasses") steer the diffusion process toward specific semantic attributes without violating the physical constraints of the degraded input.
Evaluation
The authors evaluated MCS on synthetic data (degraded CelebA-HQ) and real-world datasets (Wider-Test and WebPhoto-Test). Since MCS is training-free, it requires no paired training data.
Key Results
- Quantitative Performance: MCS achieved state-of-the-art results on no-reference quality metrics, scoring 68.08 on Wider-Test and 69.74 on WebPhoto-Test, outperforming both learning-based (e.g., DiffBIR) and sampling-based (e.g., SSDiff) methods.
- Controllability: The method demonstrated high text-responsiveness (91% in ablation studies), successfully generating diverse variations (e.g., different ethnicities or accessories) from the same blurry input while maintaining structural fidelity.
Discussion and Conclusion
The success of MCS highlights that T2I diffusion models have distinct phases—global structure followed by fine details. By applying time-varying constraints, MCS avoids the artifacts common in other sampling-based inverse solvers.
While powerful, the method currently relies on heuristic parameters for the projection radius and can struggle with extremely fine-grained or rare semantic prompts. Future work includes developing adaptive, degradation-aware estimators to further refine the restoration process and improve text-vision alignment.
Summary updated