Introduction
Nighttime dehazing is a significantly more complex task than daytime dehazing due to the presence of multiple artificial light sources. While daytime models typically assume a single global light source, nighttime scenes are characterized by localized glow, extreme shadows, and uneven illumination. NightHazeFormer addresses these challenges through a novel two-stage transformer framework that integrates physical priors and semi-supervised learning to bridge the gap between synthetic training and real-world application.
Methodology
The NightHazeFormer framework operates in two distinct phases to handle the unique degradations of nighttime photography.
Supervised Pre-training with Prior Queries
The core architecture is an encoder-decoder transformer utilizing NAFBlocks (Non-Aligned Feature blocks). A key innovation is the Prior Query Transformer in the decoder, which injects classical physical priors—the Dark Channel Prior (DCP) and the Bright Channel Prior (BCP)—as explicit queries. These priors act as guides for the self-attention mechanism, helping the model focus on hazy regions (via DCP) while maintaining contrast and brightness (via BCP).
Semi-supervised Fine-tuning
To adapt to real-world domain shifts, the model undergoes a semi-supervised refinement stage. Using unlabeled real-world hazy images, the system generates "pseudo ground truths" via an unsupervised dehazing method (BCCR). These pseudo-labels facilitate a cyclic supervised fine-tuning process, allowing the network to learn the specific distribution of real-world nighttime degradations without requiring expensive, manually labeled real-world pairs.
Datasets: Simulating Real-World Complexity
The authors introduced the UNREAL-NH dataset, a large-scale synthetic collection of 10,080 image pairs created using Unreal Engine 4.27. This dataset simulates complex optical effects such as bloom, motion blur, and lens flares that are often ignored in simpler mathematical haze models.
Statistical evaluations using t-SNE maps and Fréchet Inception Distance (FID) confirm that UNREAL-NH is significantly more representative of real-world data than existing benchmarks like NHC or GTA5. For instance, UNREAL-NH achieved an FID of 12.94, the lowest among all synthetic sets compared.
Experimental Results
NightHazeFormer demonstrates superior performance in both quantitative metrics and visual quality across various benchmarks.
Quantitative Performance
On the UNREAL-NH benchmark, NightHazeFormer achieved a PSNR of 27.92 dB, a substantial improvement over the previous state-of-the-art learning-based method, CFEN-ViT, which scored 19.17 dB.
Qualitative Analysis
Visually, the model effectively suppresses the "halo" or glow effects around streetlights, a common failure point for other methods. It restores sharp details in dark areas without introducing the color shifts or excessive darkening seen in earlier approaches. The model also performed best on real-world images (REAL-NH) according to non-reference metrics like NIQE and MUSIQ.
Conclusion
NightHazeFormer represents a significant advancement in nighttime image restoration. By leveraging physical priors as transformer queries and utilizing a semi-supervised adaptation strategy, the model successfully navigates the complexities of artificial lighting and haze. The accompanying UNREAL-NH dataset provides a robust foundation for future developments in the field, offering a more realistic benchmark for synthetic-to-real generalization.
Summary updated