Proxies for Distortion and Consistency with Applications for Real-World Image Restoration

Sean Man, Guy Ohayon, Ron Raphaeli, Michael EladarXiv 2025 · DOI · arXiv

Introduction

Restoring real-world images is a "blind" challenge because we rarely know exactly how an image was degraded or what the original looked like. Traditional restoration assumes a known forward model and evaluates success against a ground-truth image using metrics like Mean Squared Error (MSE). Real-world images, however, lack these "clean twins," creating a design gap in guiding algorithms and an evaluation gap in ranking performance. This paper introduces a suite of tools—including a degradation estimator and no-reference proxy metrics—that allow researchers to perform high-quality restoration and evaluate performance without needing ground-truth images.

Methods

The proposed framework addresses these gaps through three main pillars:

  1. Degradation Estimator: A model using a ConvNext-Large backbone predicts four parameters from a single degraded image: blur level, downsampling factor, noise level, and JPEG quality.
  2. ELAD (Empirical Likelihood Approximation with Diffusion): A new "plug-and-play" restoration algorithm. It utilizes a pre-trained diffusion model guided by the degradation estimator to ensure the restored image remains consistent with the original input pixels.
  3. Proxy Distortion Measures (ProxMSE & ProxLPIPS): These are mathematically derived "proxies" for standard metrics. By comparing a restored image to a predicted "posterior mean" (the best guess of the clean image given the data), researchers can rank algorithms accurately without ground truth.
The proposed framework for blind image restoration and evaluation.

Key Insights

A common failure in AI restoration is "hallucination"—creating a plausible but inaccurate face. By approximating the "likelihood," the authors force the model to stay consistent with the degraded input. A major breakthrough for evaluation is the realization that while MSE cannot be calculated without ground truth, the distance to the Minimum Mean Squared Error (MMSE) estimate is mathematically linked to the distance to the true ground truth. This allows for reliable algorithm ranking in real-world scenarios.

Furthermore, by running their estimator over entire datasets, the authors performed "dataset fingerprinting." They discovered, for instance, that LFW images are generally less blurry but more aggressively JPEG-compressed than WebPhoto or WIDER images.

Evaluation and Results

The study utilized CelebA-Test for synthetic validation and real-world benchmarks like LFW-Test, WebPhoto-Test, and WIDER-Test.

Dataset parameter distributions (left) and validation of proxy consistency measures (right).

Key results include:

  • Estimator Accuracy: The degradation estimator achieved high correlation scores ($R^2$) on synthetic data, particularly for blur (0.95) and scale factors (0.99).
  • ELAD Performance: The ELAD method consistently outperformed previous state-of-the-art plug-and-play methods (like DifFace and PGDiff) in both consistency (ProxCMSE) and distortion (ProxMSE).
  • Proxy Reliability: Validation experiments showed a near-perfect linear relationship between proxy metrics and their true counterparts. This confirms that researchers can finally trust no-reference rankings for real-world restoration tasks.

Limitations and Future Work

The current degradation model is limited to a specific chain (Blur $\rightarrow$ Downsample $\rightarrow$ Noise $\rightarrow$ JPEG) and does not yet account for complex effects like haze or motion-specific artifacts. Additionally, proxy metrics rely on a high-quality MMSE estimator. Future work aims to extend this framework to general imagery and other tasks like blind super-resolution (BSR).

Summary updated

Abstract

Real-world image restoration deals with the recovery of images suffering from an unknown degradation. This task is typically addressed while being given only degraded images, without their corresponding ground-truth versions. In this hard setting, designing and evaluating restoration algorithms becomes highly challenging. This paper offers a suite of tools that can serve both the design and assessment of real-world image restoration algorithms. Our work starts by proposing a trained model that predicts the chain of degradations a given real-world measured input has gone through. We show how this estimator can be used to approximate the consistency -- the match between the measurements and any proposed recovered image. We also use this estimator as a guiding force for the design of a simple and highly-effective plug-and-play real-world image restoration algorithm, leveraging a pre-trained diffusion-based image prior. Furthermore, this work proposes no-reference proxy measures of MSE and LPIPS, which, without access to the ground-truth images, allow ranking of real-world image restoration algorithms according to their (approximate) MSE and LPIPS. The proposed suite provides a versatile, first of its kind framework for evaluating and comparing blind image restoration algorithms in real-world scenarios.
BibTeX
@misc{man2025proxies,
  author = {Sean Man and Guy Ohayon and Ron Raphaeli and Michael Elad},
  title = {{Proxies for Distortion and Consistency with Applications for Real-World Image Restoration}},
  year = {2025},
  eprint = {2501.12102},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2501.12102},
}
Figure