Extreme Blind Image Restoration via Prompt-Conditioned Information Bottleneck

Hongeun Kim, Bryan Sangwoo Kim, Jong Chul Ye

Abstract

Blind Image Restoration (BIR) methods have achieved remarkable success but falter when faced with Extreme Blind Image Restoration (EBIR), where inputs suffer from severe, compounded degradations beyond their training scope. Directly learning a mapping from extremely low-quality (ELQ) to high-quality (HQ) images is challenging due to the massive domain gap, often leading to unnatural artifacts and loss of detail. To address this, we propose a novel framework that decomposes the intractable ELQ-to-HQ restoration process. We first learn a projector that maps an ELQ image onto an intermediate, less-degraded LQ manifold. This intermediate image is then restored to HQ using a frozen, off-the-shelf BIR model. Our approach is grounded in information theory; we provide a novel perspective of image restoration as an Information Bottleneck problem and derive a theoretically-driven objective to train our projector. This loss function effectively stabilizes training by balancing a low-quality reconstruction term with a high-quality prior-matching term. Our framework enables Look Forward Once (LFO) for inference-time prompt refinement, and supports plug-and-play strengthening of existing image restoration models without need for finetuning. Extensive experiments under severe degradation regimes provide a thorough analysis of the effectiveness of our work.

Introduction

Blind Image Restoration (BIR) aims to recover a clean image from a degraded version without prior knowledge of the specific degradation process (e.g., the exact combination of blur, noise, and compression). While modern deep learning models have made significant strides in standard BIR, they often fail catastrophically when faced with Extreme Blind Image Restoration (EBIR). In EBIR scenarios, the degradations are so severe that they fall far outside the distribution of the model's training data. This massive domain gap typically results in "hallucinations"—where the model generates realistic-looking textures that bear no structural resemblance to the original scene—or severe artifacts that render the output unusable.

The Challenge of Extreme Degradation

Existing restoration models generally attempt to map an extremely low-quality (ELQ) input directly to a high-quality (HQ) output in a single step. Researchers from KAIST AI identify this as an "intractable" problem because the solution space for such a massive jump is too large for the model to navigate reliably. When the input is nearly unrecognizable, the model lacks sufficient structural anchors to reconstruct the original image faithfully.

Visual comparison between extreme low-quality (ELQ) inputs and high-quality (HQ) targets.

Methodology: Decomposition and Information Theory

The core innovation of this work is the decomposition of the restoration process into two manageable steps: $ELQ \to LQ \to HQ$. Instead of attempting the full leap at once, the framework first projects the ELQ image onto a standard Low-Quality (LQ) manifold—essentially "cleaning up" the most severe noise and artifacts to reach a state that standard restoration models can handle.

The Information Bottleneck Perspective

The authors reframe image restoration through the lens of Information Bottleneck (IB) theory. In this view, the restored image acts as a compressed representation of the input that retains only relevant scene information while discarding degradation noise. This theoretical grounding leads to the Image Restoration Information Bottleneck (IRIB) loss function, which balances two critical objectives:

  1. LQ Reconstruction: Ensuring the intermediate projected image matches a moderately degraded version of the ground truth.
  2. HQ Prior-Matching: Utilizing a diffusion-based loss to ensure the final output aligns with the distribution of real, high-quality images.
Conceptual diagrams comparing the baseline approach with the proposed Information Bottleneck and Look Forward Once frameworks.

Look Forward Once (LFO)

Modern restoration models often rely on text prompts (e.g., "a photo of a building") to guide the reconstruction. However, generating an accurate prompt from a garbled ELQ image is difficult. To solve this, the authors introduce Look Forward Once (LFO). This inference-time refinement uses the initial intermediate LQ sample to generate a more accurate text prompt via a Vision-Language Model. This refined prompt is then fed back into the model for a second, more precise restoration pass, significantly reducing hallucinations.

Experimental Results

The framework was trained on the LSDIR dataset and evaluated on DIV2K and DIV8K using a modified Real-ESRGAN pipeline to simulate extreme conditions.

Quantitative Performance

On the DIV2K dataset, the IRIB framework achieved a state-of-the-art FID score of 62.9, a substantial improvement over the baseline OSEDiff score of 81.5 (where lower is better). Furthermore, the framework is "plug-and-play." When added as a front-end projector to the existing SeeSR model, it improved SeeSR's FID from 69.3 to 61.2 without requiring any finetuning of the SeeSR model itself.

Qualitative Impact

Visually, the method excels at maintaining global structure. While standard models might produce "melted" architectural features or unrecognizable animal faces under extreme noise, the IRIB framework recovers sharp edges, realistic textures, and correct structural details, such as window frames and eye shapes.

Conclusion

By decomposing the restoration process and grounding it in Information Bottleneck theory, this research provides a robust solution for the most challenging image restoration tasks. The ability to enhance existing, frozen models through a simple projector makes this a highly versatile approach for real-world applications where degradations are unpredictable and severe. Future work may focus on reducing the computational cost of backpropagating through multi-step diffusion backbones and improving the robustness of prompt extraction modules.

Summary updated

Abstract

Blind Image Restoration (BIR) methods have achieved remarkable success but falter when faced with Extreme Blind Image Restoration (EBIR), where inputs suffer from severe, compounded degradations beyond their training scope. Directly learning a mapping from extremely low-quality (ELQ) to high-quality (HQ) images is challenging due to the massive domain gap, often leading to unnatural artifacts and loss of detail. To address this, we propose a novel framework that decomposes the intractable ELQ-to-HQ restoration process. We first learn a projector that maps an ELQ image onto an intermediate, less-degraded LQ manifold. This intermediate image is then restored to HQ using a frozen, off-the-shelf BIR model. Our approach is grounded in information theory; we provide a novel perspective of image restoration as an Information Bottleneck problem and derive a theoretically-driven objective to train our projector. This loss function effectively stabilizes training by balancing a low-quality reconstruction term with a high-quality prior-matching term. Our framework enables Look Forward Once (LFO) for inference-time prompt refinement, and supports plug-and-play strengthening of existing image restoration models without need for finetuning. Extensive experiments under severe degradation regimes provide a thorough analysis of the effectiveness of our work.
BibTeX
@misc{kim2025extreme,
  author = {Hongeun Kim and Bryan Sangwoo Kim and Jong Chul Ye},
  title = {{Extreme Blind Image Restoration via Prompt-Conditioned Information Bottleneck}},
  year = {2025},
  eprint = {2510.00728},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2510.00728},
}
Figure