Structure-Guided Diffusion Transformer for Low-Light Image Enhancement

Xiangchen Yin, Zhenda Yu, Longtao Jiang, Xin Gao, Xiao Sun, Zhi Liu, Xun Yang

Abstract

Low-light image enhancement (LLIE) remains a significant challenge due to the simultaneous need for illumination recovery and noise suppression. This paper introduces SDTL, the first framework to leverage Diffusion Transformers (DiT) for this task. By integrating wavelet-based feature compression with a novel structure-guided attention mechanism, SDTL achieves state-of-the-art performance, effectively restoring fine textures while avoiding the artifacts common in traditional diffusion models.

Introduction

Capturing high-quality images in low-light environments is difficult because brightening a dark scene inherently amplifies sensor noise. Traditional deep learning approaches, such as CNNs and standard Transformers, often struggle with this trade-off, resulting in either excessive noise or over-smoothed details.

While Denoising Diffusion Probabilistic Models (DDPMs) have shown promise in generating realistic textures, their computational cost and potential instability in restoration tasks have limited their adoption. SDTL addresses these gaps by replacing the standard U-Net backbone of diffusion models with a scalable Diffusion Transformer (DiT) architecture, specifically optimized for the unique constraints of low-light enhancement.

Methodology

The SDTL framework is built upon three core architectural innovations designed for efficiency and structural integrity:

  1. Wavelet-Based Efficiency: To reduce computational overhead, SDTL operates in the frequency domain. It uses a Discrete Wavelet Transform (DWT) to decompose the input image into low-frequency (global layout) and high-frequency (texture and noise) subbands. This allows the model to process compressed features and focus on specific directional frequency components.
  2. Structure Enhancement Module (SEM): This module utilizes structural priors—such as edge and layout information—to guide the restoration of high-frequency details. It employs a bi-directional guidance mechanism to reconstruct missing structural data in dark regions and an adaptive fusion strategy to synthesize the final enhanced features.
  3. Structure-guided Attention Block (SAB): Within the DiT backbone, standard self-attention is replaced by the SAB. This block uses structural tokens to direct the model's focus toward texture-rich areas, ensuring that the denoising process preserves critical details while ignoring featureless, noisy regions.
The SDTL architecture, highlighting the wavelet decomposition, the Structure Enhancement Module (SEM), and the Denoising DiT backbone.

Experiments and Results

SDTL was evaluated on several benchmark datasets, including LOLv1, LOLv2-real, and LSRW for quantitative assessment, and LIME and VV for "in the wild" testing.

Quantitative Performance

On the LOLv1 dataset, SDTL established a new state-of-the-art benchmark, achieving a PSNR of 27.34 dB and an SSIM of 0.862. As shown in the performance comparison below, SDTL significantly outperforms recent competitive models like DiffLL and Retinexformer in both signal-to-noise ratio and structural similarity.

Quantitative comparison of SDTL (Ours) against state-of-the-art methods on the LOLv1 dataset, showing superior PSNR and SSIM.

Qualitative and Downstream Impact

Visual results demonstrate that SDTL restores natural colors and sharp textures where other models produce "muddy" artifacts. Beyond visual quality, the model's utility was proven in downstream tasks. When used as a pre-processor for nighttime semantic segmentation on the ACDC dataset, SDTL improved the Mean Intersection over Union (mIoU) by 2.68% over the baseline, highlighting its potential for autonomous driving and surveillance applications.

Discussion and Conclusion

SDTL demonstrates that the scalability and long-range dependency modeling of Transformers make them superior to traditional U-Nets for diffusion-based image restoration. By treating structure as a "compass," the model successfully distinguishes between fine texture and sensor noise.

Despite its success, the iterative nature of diffusion models remains more computationally intensive than single-pass architectures. Future work will focus on optimizing inference speed and extending the structure-guided diffusion approach to video enhancement and 3D engineering applications. In summary, SDTL represents a significant step forward in applying generative diffusion models to practical, high-precision image restoration tasks.

Summary updated

Abstract

While the diffusion transformer (DiT) has become a focal point of interest in recent years, its application in low-light image enhancement remains a blank area for exploration. Current methods recover the details from low-light images while inevitably amplifying the noise in images, resulting in poor visual quality. In this paper, we firstly introduce DiT into the low-light enhancement task and design a novel Structure-guided Diffusion Transformer based Low-light image enhancement (SDTL) framework. We compress the feature through wavelet transform to improve the inference efficiency of the model and capture the multi-directional frequency band. Then we propose a Structure Enhancement Module (SEM) that uses structural prior to enhance the texture and leverages an adaptive fusion strategy to achieve more accurate enhancement effect. In Addition, we propose a Structure-guided Attention Block (SAB) to pay more attention to texture-riched tokens and avoid interference from noisy areas in noise prediction. Extensive qualitative and quantitative experiments demonstrate that our method achieves SOTA performance on several popular datasets, validating the effectiveness of SDTL in improving image quality and the potential of DiT in low-light enhancement tasks.
BibTeX
@article{yin2025structure,
  author = {Xiangchen Yin and Zhenda Yu and Longtao Jiang and Xin Gao and Xiao Sun and Zhi Liu and Xun Yang},
  title = {{Structure-Guided Diffusion Transformer for Low-Light Image Enhancement}},
  journal = {IEEE Trans. Multimedia},
  year = {2025},
  doi = {10.1109/tmm.2025.3613117},
}
Figure