Double Domain Guided Real-Time Low-Light Image Enhancement for Ultra-High-Definition Transportation Surveillance

Jingxiang Qu, Ryan Wen Liu, Yuan Gao, Yu Guo, Fenghua Zhu, Fei-Yue Wang

TL;DR

This paper introduces DDNet, a deep learning framework designed to enhance 4K (UHD) low-light surveillance footage in real-time. By processing images simultaneously in both the color and gradient domains, the network restores visibility and sharpens edge details—like license plates and pedestrians—at over 37 frames per second, significantly outperforming existing methods in both speed and visual quality.

What Problem Does This Paper Address?

Intelligent Transportation Systems (ITS) increasingly rely on Ultra-High-Definition (UHD) cameras for tasks like traffic monitoring and accident detection. However, at night or in low-light conditions, these cameras produce noisy, low-contrast images where critical details (edges of vehicles, lane markings) disappear.

Existing enhancement algorithms face a "speed vs. quality" wall:

  1. Computational Cost: Processing 4K pixels is extremely heavy; many models require resizing images down, which destroys fine details.
  2. Noise vs. Detail: Standard methods often amplify sensor noise while trying to brighten the image, or blur out edges while trying to denoise.
  3. Downstream Failure: Poorly enhanced images lead to failures in AI tasks like object detection and scene segmentation.

Methods

The authors propose the Double Domain Guided Network (DDNet), which uses an encoder-decoder structure featuring several novel components:

  • Double Domain Processing: Instead of just brightening pixels, the network splits the task. The Coarse Enhancement Module (CEM) focuses on color and brightness, while the Gradient Enhancement Module (GEM) specifically targets edges and textures.
  • LoG-based Guidance: To make the gradient enhancement robust to noise, they use a Laplacian of Gaussian (LoG) operator. Unlike a standard Laplacian filter, LoG smooths the image first to prevent the network from "enhancing" random noise as if it were a real edge.
  • ScCAM (Self-calibrated Convolutions with Attention Module): To keep the model fast enough for real-time 4K, they designed a lightweight attention mechanism. It helps the model focus on "valuable" parts of the image (like a distant car) without the massive overhead of standard Transformers or heavy CNNs.
  • Joint Loss Function: The model is trained using a multi-part loss that forces it to be accurate in color, consistent in gradients, and structurally similar to the ground truth.

Datasets

The researchers evaluated DDNet using a mix of standard and domain-specific data:

  • LOL Dataset: A standard benchmark for low-light enhancement containing 1,500 image pairs.
  • Synthetic Transportation Data: 1,000 outdoor images from PASCAL VOC, COCO, and DETRAC, artificially darkened to simulate night-time surveillance.
  • UHD Test Sets: Representative low-light images from DICM, LIME, MEF, and TMDIED.
  • Real-Captured UHD Surveillance: The authors collected and released their own dataset of raw 4K low-light transportation footage to prove real-world viability.

Key Insights and Ideas

  • Gradients are the "Aha!" Moment: In transportation, a "pretty" image is less important than a "sharp" one. By treating the gradient domain as a first-class citizen, DDNet preserves the high-frequency details necessary for reading license plates.
  • Efficiency via Architecture, not Resizing: Most 4K solutions involve downsampling. DDNet proves that by using "self-calibrated" convolutions, you can process full-resolution 4K data directly while maintaining a high frame rate.
  • Noise Pre-filtering: Integrating Gaussian smoothing into the gradient extraction (the "G" in LoG) is a simple but effective way to stop the "salt and pepper" noise common in low-light deep learning outputs.

Key Results

  • Speed: DDNet processes 4K (3840 × 2160) images in 0.027 seconds (~37 FPS) on an Nvidia A40 GPU. For comparison, many state-of-the-art models take over a second per frame or cannot process 4K at all.
  • Image Quality: On the LOL dataset, DDNet achieved a PSNR of 21.86 and SSIM of 0.832, outperforming nearly 20 other recent methods including KinD, EnlightenGAN, and LLFlow.
  • AI Performance: Using DDNet as a pre-processor improved object detection accuracy (YOLOv5/X) and scene segmentation (DAFormer) significantly compared to using raw low-light footage or other enhancement methods.

Limitations and Future Work

  • Data Scarcity: The authors note that capturing perfectly paired "low-light" and "normal-light" 4K images in the real world is nearly impossible. They suggest exploring Semi-supervised learning or GANs to reduce reliance on paired data.
  • Model Size: While fast, the model isn't yet "ultra-lightweight." Future work includes using model pruning to make it run on edge devices with less VRAM.
  • Motion Blur: The current model focuses on static enhancement. Fast-moving vehicles on expressways still suffer from motion blur, which the authors plan to address via multi-task learning for deblurring.

Summary updated

Abstract

Real-time transportation surveillance is an essential part of the intelligent transportation system (ITS). However, images captured under low-light conditions often suffer poor visibility with types of degradation, such as noise interference and vague edge features, etc. With the development of imaging devices, the quality of the visual surveillance data is continually increasing, like 2K and 4K, which have more strict requirements on the efficiency of image processing. To satisfy the requirements on both enhancement quality and computational speed, this paper proposes a double domain guided real-time low-light image enhancement network (DDNet) for ultra-high-definition (UHD) transportation surveillance. Specifically, we design an encoder-decoder structure as the main architecture of the learning network. In particular, the enhancement processing is divided into two subtasks (i.e., color enhancement and gradient enhancement) via the proposed coarse enhancement module (CEM) and LoG-based gradient enhancement module (GEM), which are embedded in the encoder-decoder structure. It enables the network to enhance the color and edge features simultaneously. Through the decomposition and reconstruction on both color and gradient domains, our DDNet can restore the detailed feature information concealed by the darkness with better visual quality and efficiency. The evaluation experiments on standard and transportation-related datasets demonstrate that our DDNet provides superior enhancement quality and efficiency compared with state-of-the-art methods. Besides, the object detection and scene segmentation experiments indicate the practical benefits for higher-level image analysis under low-light environments in ITS. The source code is available at https://github.com/QuJX/DDNet.
BibTeX
@article{qu2024double,
  author = {Jingxiang Qu and Ryan Wen Liu and Yuan Gao and Yu Guo and Fenghua Zhu and Fei-Yue Wang},
  title = {{Double Domain Guided Real-Time Low-Light Image Enhancement for Ultra-High-Definition Transportation Surveillance}},
  journal = {IEEE Transactions on Intelligent Transportation Systems},
  year = {2024},
  url = {https://openreview.net/forum?id=20XuqdYnoc},
}
Figure