AWRaCLe: All-Weather Image Restoration Using Visual In-Context Learning

Sudarshan Rajagopalan, Vishal M. Patel

Abstract

All-Weather Image Restoration (AWIR) aims to remove environmental artifacts such as rain, snow, and haze. Traditional models often struggle with generalization across different weather types and the disentanglement of degradation from the underlying scene. We present AWRaCLe, a novel framework that leverages visual in-context learning. By utilizing a reference "context pair"—a degraded image and its clean counterpart—AWRaCLe extracts precise degradation "fingerprints" to guide the restoration of query images. Our approach achieves state-of-the-art performance across multiple benchmarks and demonstrates unique capabilities like selective restoration.

Introduction

Environmental conditions like rain streaks, snowflakes, and thick haze significantly degrade image quality, impacting downstream computer vision tasks. While deep learning has advanced All-Weather Image Restoration (AWIR), two primary challenges remain: generalization and disentanglement. "All-in-one" models often lack the specific guidance required for the unique textures of various weather types. Furthermore, isolating weather artifacts from the background scene in a single image is mathematically ill-posed.

Previous attempts to provide guidance used text prompts (e.g., "remove rain"), but text lacks the spatial resolution to describe complex weather patterns. AWRaCLe addresses this by using visual in-context learning, where a context pair (degraded and clean versions of the same scene) provides high-fidelity degradation information.

Visual in-context learning examples showing dehazing, desnowing, deraining, and selective restoration.

Proposed Method: AWRaCLe

The AWRaCLe architecture is designed to extract and inject degradation-specific information (DSI) into a restoration backbone.

Architecture Overview

The system consists of three primary components:

  1. Restoration Backbone: A modified Restormer architecture serves as the high-performance transformer-based engine for image processing.
  2. Degradation Context Extraction (DCE) Blocks: These blocks utilize a frozen CLIP image encoder to obtain high-level features from the context pair. Multi-Head Self-Attention (MHSA) is then applied to compare the degraded and clean context images. Since the underlying scene is identical, the attention mechanism effectively isolates the weather artifacts.
  3. Context Fusion (CF) Blocks: These blocks employ Multi-Head Cross-Attention (MHCA) to integrate the extracted weather "fingerprint" into the restoration network. By injecting this information at multiple levels of the decoder, the model maintains precise guidance throughout the restoration process.
The AWRaCLe architecture featuring the Restormer backbone, DCE blocks, and CF blocks.

Experiments and Results

AWRaCLe was evaluated on standard benchmarks including Snow100k, SRD (rain), and RESIDE (haze). It achieved state-of-the-art (SOTA) results, notably reaching a PSNR of 31.7 dB on the SOTS haze dataset and 35.7 dB on Rain100L.

Key Insights

  • Visual vs. Textual Context: Visual examples provide significantly richer DSI than text prompts, especially for describing the transparency and spatial density of haze or rain.
  • Selective Restoration: A unique property of AWRaCLe is its ability to follow specific context instructions. If a query image contains both haze and snow, providing a "haze-only" context pair results in the removal of haze while leaving the snow intact.
  • Robustness and Generalization: The model demonstrates strong performance on out-of-distribution (OoD) datasets (e.g., Foggy Cityscapes), indicating that the in-context learning approach generalizes better than standard supervised learning.

Conclusion and Future Work

AWRaCLe introduces a powerful paradigm for image restoration by treating weather removal as an in-context learning task. While the model requires a known degradation type to select the appropriate context pair, future work could involve automating this selection. Additionally, exploring multi-context pairs for compound weather events remains an open area for research.

Summary updated

Abstract

All-Weather Image Restoration (AWIR) under adverse weather conditions is a challenging task due to the presence of different types of degradations. Prior research in this domain relies on extensive training data but lacks the utilization of additional contextual information for restoration guidance. Consequently, the performance of existing methods is limited by the degradation cues that are learnt from individual training samples. Recent advancements in visual in-context learning have introduced generalist models that are capable of addressing multiple computer vision tasks simultaneously by using the information present in the provided context as a prior. In this paper, we propose All-Weather Image Restoration using Visual In-Context Learning (AWRaCLe), a novel approach for AWIR that innovatively utilizes degradation-specific visual context information to steer the image restoration process. To achieve this, AWRaCLe incorporates Degradation Context Extraction (DCE) and Context Fusion (CF) to seamlessly integrate degradation-specific features from the context into an image restoration network. The proposed DCE and CF blocks leverage CLIP features and incorporate attention mechanisms to adeptly learn and fuse contextual information. These blocks are specifically designed for visual in-context learning under all-weather conditions and are crucial for effective context utilization. Through extensive experiments, we demonstrate the effectiveness of AWRaCLe for all-weather restoration and show that our method advances the state-of-the-art in AWIR.
BibTeX
@inproceedings{rajagopalan2025awracle,
  author = {Sudarshan Rajagopalan and Vishal M. Patel},
  title = {{AWRaCLe: All-Weather Image Restoration Using Visual In-Context Learning}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2025},
  doi = {10.1609/aaai.v39i6.32716},
}
Figure