EvoIR: Towards All-in-One Image Restoration via Evolutionary Frequency Modulation

Jiaqi Ma, Shengkai Hu, Xu Zhang, Jun Wan, Jiaxing Huang, Lefei Zhang, Salman Khan

Abstract

Image restoration aims to recover high-quality images from degraded versions suffering from noise, haze, rain, or blur. Traditional methods often rely on task-specific models, which struggle with real-world scenarios involving composite degradations. We present EvoIR, an "All-in-One" image restoration framework that addresses two critical bottlenecks: spatial blindness and static training priorities. By introducing a Frequency-Modulated Module (FMM) for targeted spectral processing and an Evolutionary Optimization Strategy (EOS) for dynamic loss balancing, EvoIR achieves state-of-the-art performance across multiple benchmarks with minimal computational overhead.

Introduction

All-in-One Image Restoration (AiOIR) seeks to develop a single unified model capable of handling diverse degradation types. However, existing AiOIR models often ignore the distinct frequency characteristics of different degradations—for instance, noise typically resides in high-frequency bands while structural information is low-frequency. Furthermore, training a single model on multiple tasks simultaneously often leads to "gradient conflicts," where the optimization path for one task (e.g., deraining) hinders another (e.g., low-light enhancement). EvoIR overcomes these challenges by explicitly separating image signals in the frequency domain and employing a biologically inspired evolutionary strategy to navigate complex multi-task optimization landscapes.

Proposed Method

Frequency-Modulated Module (FMM)

The core of the EvoIR architecture is the Frequency-Modulated Module (FMM). Unlike standard spatial-only convolutions, the FMM splits features into two distinct paths:

  • Low-Frequency Path: Utilizes Fast Fourier Transforms (FFT) and spectral gating to suppress noise-heavy bands while preserving the global structural integrity of the image.
  • High-Frequency Path: Operates in the spatial domain using learned masks to identify and sharpen edges, textures, and fine details.

Evolutionary Optimization Strategy (EOS)

To mitigate gradient conflicts during multi-task training, EvoIR employs an Evolutionary Optimization Strategy (EOS). Instead of using fixed loss weights, the system maintains a population of weight configurations. Through iterative cycles of fitness testing, selection, crossover, and mutation, the model automatically discovers the optimal balance for various training goals. Remarkably, this sophisticated balancing act is highly efficient, accounting for only 0.1% of the total training time per epoch.

The EvoIR framework architecture, highlighting the Evolutionary Optimization Strategy (EOS) for loss balancing and the Frequency-Modulated Module (FMM) for dual-path feature processing.

Experimental Results

Quantitative Performance

EvoIR was evaluated against several state-of-the-art (SOTA) models across standard restoration benchmarks and composite degradation datasets. In the 3-task setting (Denoising, Dehazing, and Deraining), EvoIR achieved a Peak Signal-to-Noise Ratio (PSNR) of 33.00 dB, surpassing the previous leading method, DFPIR. In the more complex 5-task setting (adding Deblurring and Low-light enhancement), EvoIR maintained its dominance with a PSNR of 30.83 dB.

Performance comparison of EvoIR against state-of-the-art methods across 3-task (3D) and 5-task (5D) restoration benchmarks.

Remote Sensing and Efficiency

The model's robustness was further validated on the MDRS-Landsat dataset for satellite imagery, where it outperformed specialized models like Ada4DIR-d despite having fewer parameters (37M vs 41M). Ablation studies confirmed that placing FMMs in the earlier stages of the network is critical for capturing frequency-aware features that benefit downstream restoration.

Discussion and Conclusion

EvoIR demonstrates that frequency-aware architectural design combined with adaptive evolutionary training provides a powerful solution for All-in-One image restoration. While the model sets new SOTA records in most categories, deblurring remains a challenging area where specialized spatial-frequency kernels may be required for further improvement. Future research will explore the integration of these frequency-modulated components into large-scale generative models and video restoration tasks.

Summary updated

Abstract

All-in-One Image Restoration (AiOIR) tasks often involve diverse degradation that require robust and versatile strategies. However, most existing approaches typically lack explicit frequency modeling and rely on fixed or heuristic optimization schedules, which limit the generalization across heterogeneous degradation. To address these limitations, we propose EvoIR, an AiOIR‑specific framework that introduces evolutionary frequency modulation for dynamic and adaptive image restoration. Specifically, EvoIR employs the Frequency-Modulated Module (FMM) that decomposes features into high- and low-frequency branches in an explicit manner and adaptively modulates them to enhance both structural fidelity and fine-grained details. Central to EvoIR, an Evolutionary Optimization Strategy (EOS) iteratively adjusts frequency-aware objectives through a population-based evolutionary process, dynamically balancing structural accuracy and perceptual fidelity. Its evolutionary guidance further mitigates gradient conflicts across degradation and accelerates convergence. By synergizing FMM and EOS, EvoIR yields greater improvements than using either component alone, underscoring their complementary roles. Extensive experiments on multiple benchmarks demonstrate that EvoIR outperforms state-of-the-art AiOIR methods.
BibTeX
@misc{ma2025evoir,
  author = {Jiaqi Ma and Shengkai Hu and Xu Zhang and Jun Wan and Jiaxing Huang and Lefei Zhang and Salman Khan},
  title = {{EvoIR: Towards All-in-One Image Restoration via Evolutionary Frequency Modulation}},
  year = {2025},
  eprint = {2512.05104},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2512.05104},
}
Figure