Low-Light Image Enhancement via CLIP-Fourier Guided Wavelet Diffusion

Minglong Xue, Jinhong He, Wenhai Wang, Mingliang Zhou

Low-light image enhancement (LLIE) is a classic computer vision challenge: how do you recover color, contrast, and detail from an underexposed image without introducing a mess of noise or "hallucinated" artifacts? While deep learning has made massive strides, many models still struggle with color distortion or lose fine-grained textures in the shadows.

A new paper, "Low-light Image Enhancement via CLIP-Fourier Guided Wavelet Diffusion" (CFWD), proposes a sophisticated solution. By combining the generative power of diffusion models with multimodal semantic guidance (CLIP) and frequency-domain constraints (Fourier and Wavelets), the authors have set a new state-of-the-art for making the dark visible.

TL;DR

The CFWD framework performs image diffusion in the wavelet domain to save computation while using CLIP-based semantic prompts and Fourier-domain constraints to ensure the results are both visually natural and structurally accurate. It effectively bridges the gap between "degraded" and "normal" images, outperforming existing methods on major benchmarks like LOL and LSRW.

What Problem Does This Paper Address?

Traditional low-light enhancement often relies on manually designed priors (like the Retinex theory), which struggle to adapt to diverse real-world lighting. Modern deep learning methods (CNNs and Transformers) often supervise the process only at the image level. This "unimodal" approach frequently fails to reconstruct local structures or results in "diversity confusion"—where the model isn't quite sure what the final textures should look like, leading to noise or color shifts.

Furthermore, while Diffusion Models are excellent at generating high-quality images, they are computationally expensive and can be "too creative," sometimes adding redundant or confusing information that wasn't in the original scene.

Methods

The authors introduce CFWD, which stands on three main pillars:

  1. Wavelet Diffusion Model: Instead of running the diffusion process on high-resolution pixels, the model applies a Discrete Wavelet Transform (DWT). It performs the noisy-to-clean iteration only on the low-frequency subband. This significantly reduces the computational load (spatial dimensions are cut to 1/4) while preserving the core image content.
  2. Multiscale Visual-Language Guidance (CLIP): This is the "secret sauce." The model uses the pre-trained CLIP model to align image features with text prompts. It uses positive prompts (e.g., "clear, bright") and negative prompts (e.g., "dark, noisy") to "pull" the enhancement toward a high-quality result and "push" it away from degradation artifacts.
  3. Hybrid High Frequency Perception Module (HFPM): To ensure the fine details (high frequencies) aren't lost, the authors combine Wavelet and Fourier transforms. By minimizing the difference in the amplitude and phase of the high-frequency components between the enhanced image and the ground truth, the model achieves much sharper structural recovery.

Datasets

The model was rigorously tested on several real-world and high-resolution datasets:

  • Paired Datasets: LOLv1, LOLv2-Real, and LSRW (containing thousands of low/normal light pairs).
  • High-Resolution Benchmark: The BAID dataset, featuring 2K resolution backlit images.
  • Unpaired/Generalization Sets: LIME and DICM, used to test how the model performs on "in-the-wild" images without ground truth.

Key Insights and Ideas

  • Semantics Matter for Restoration: Usually, CLIP is used for high-level tasks like image generation or classification. Here, the authors show that semantic "anchors" (knowing what a "bright" image should look like) provide a powerful constraint that prevents the model from drifting into unrealistic color or noise patterns.
  • Frequency Synergy: Wavelets provide spatial-frequency localization (great for denoising), while the Fourier transform captures global structural patterns. Using both helps the model distinguish between actual image details and random sensor noise.
  • Efficiency via Wavelets: By shifting the heavy lifting of diffusion to the wavelet domain, the model becomes practical for higher-resolution tasks (like 2K or 4K) that would otherwise crash standard pixel-space diffusion models.

Key Results

CFWD consistently outperformed SOTA (State-of-the-Art) methods across the board:

  • Quantitative: On the LOLv1 dataset, it achieved a PSNR of 29.185 dB, a significant jump over previous leaders like GSAD (27.629 dB).
  • Perceptual: It scored the lowest (best) on LPIPS and FID metrics, indicating that the results look more "human-natural" and have fewer artifacts.
  • High-Res Performance: On 2K resolution images (BAID), it maintained a PSNR of 26.918 dB, proving its robustness for modern photography standards.
  • Qualitative: Visual comparisons show that while other models often overexpose certain areas or leave "color clouds" in the shadows, CFWD produces clean, balanced, and sharp results.

Limitations and Future Work

Despite the impressive results, the authors acknowledge a few hurdles:

  • Computational Overhead: Even in the wavelet domain, diffusion models are slower than simple CNN-based enhancers.
  • Prompt Complexity: The performance relies on the design of the text prompts. Future work could involve more automated or "learnable" prompt engineering to make the model even more robust.
  • Redundant Content: There is still a slight risk of the model "hallucinating" details in extremely dark regions where zero information exists. The authors plan to investigate more compact loss functions to further mitigate this.

Summary updated

Abstract

Low-light image enhancement techniques have significantly progressed, but unstable image quality recovery and unsatisfactory visual perception are still significant challenges. To solve these problems, we propose a novel and robust low-light image enhancement method via CLIP-Fourier Guided Wavelet Diffusion, abbreviated as CFWD. Specifically, CFWD leverages multimodal visual-language information in the frequency domain space created by multiple wavelet transforms to guide the enhancement process. Multi-scale supervision across different modalities facilitates the alignment of image features with semantic features during the wavelet diffusion process, effectively bridging the gap between degraded and normal domains. Moreover, to further promote the effective recovery of the image details, we combine the Fourier transform based on the wavelet transform and construct a Hybrid High Frequency Perception Module (HFPM) with a significant perception of the detailed features. This module avoids the diversity confusion of the wavelet diffusion process by guiding the fine-grained structure recovery of the enhancement results to achieve favourable metric and perceptually oriented enhancement. Extensive quantitative and qualitative experiments on publicly available real-world benchmarks show that our approach outperforms existing state-of-the-art methods, achieving significant progress in image quality and noise suppression. The project code is available at https://github.com/hejh8/CFWD.
BibTeX
@article{xue2025low,
  author = {Minglong Xue and Jinhong He and Wenhai Wang and Mingliang Zhou},
  title = {{Low-Light Image Enhancement via CLIP-Fourier Guided Wavelet Diffusion}},
  journal = {ACM Trans. Multim. Comput. Commun. Appl.},
  year = {2025},
  doi = {10.1145/3764933},
}

Loading related content...

Figure