DFVO: Learning Darkness-Free Visible and Infrared Image Disentanglement and Fusion All at Once

Qi Zhou, Yukai Shi, Xiaojun Yang, Xiaoyu Xian, Lunjia Liao, Ruimao Zhang, Liang Lin

Introduction

In the realms of autonomous driving and security surveillance, the ability to perceive surroundings accurately under varying lighting conditions is paramount. Visible light sensors provide high-resolution textures and color information but are rendered nearly useless in low-light or "darkness" conditions. Conversely, infrared (IR) sensors are robust against lighting variations, capturing thermal signatures even in total darkness, yet they lack the fine-grained semantic and textural details found in visible imagery.

To bridge this gap, researchers have turned to visible-infrared image fusion (VIF). However, conventional approaches often struggle when the visible input is severely degraded by low light. This paper introduces DFVO (Darkness-Free Visible-Infrared Once), a novel framework designed to handle low-light enhancement and image fusion simultaneously within a single, unified architecture.

The Information Entropy Challenge

Most existing solutions follow a two-stage cascaded strategy: an initial enhancement network brightens the dark visible image, followed by a separate fusion network that merges the enhanced result with the IR data. While intuitive, this approach is fundamentally flawed from an information-theoretic perspective.

According to Information Entropy theory, passing data through hierarchical, non-invertible stages leads to a gradual loss of information and the amplification of noise. In the context of VIF, this "two-stage tax" often results in fused images that appear blurry or dim, suffer from over-exposure artifacts, or lose critical high-frequency details. These degradations ultimately hinder high-level vision tasks, such as pedestrian detection or obstacle avoidance.

The DFVO Architecture: Learning All at Once

DFVO addresses these issues by replacing the sequential pipeline with a Latent-Common Feature Extractor (LCFE). This module enables the network to perform three sub-tasks in parallel: infrared reconstruction, illumination disentanglement, and image fusion. By treating enhancement as a component of the fusion process rather than a prerequisite, the model retains more of the original signal.

Lossless Detail Extraction

A key innovation within DFVO is the Details-Extraction Module (DEM). Standard convolutional layers typically compress data, leading to the loss of fine textures. To combat this, the authors employ Invertible Neural Networks (INNs). Because INNs are theoretically lossless and maintain a bijective mapping of features, they allow the model to preserve fine-grained semantic information from the source images that would otherwise be discarded during downsampling.

Guided Feature Fusion

To effectively merge the modalities, the Hyper Cross-Attention Module (HCAM) utilizes a "spatial guide" approach. In low-light scenarios, the infrared image provides the most reliable structural information. The HCAM uses the IR features as "Keys" and "Values" in an attention mechanism, while the visible image provides the "Query." This effectively uses the heat signature of objects to "pull" relevant color and texture information out of the dark visible frame, ensuring that the network knows exactly where to focus even in deep shadows.

Experimental Evaluation

The authors primarily evaluated DFVO using the LLVIP dataset, which contains over 15,000 pairs of visible and infrared images captured in low-light environments. To ensure the model's robustness, it was also tested on the MSRS, SMOD, and KAIST benchmarks.

Quantitative and Qualitative Performance

DFVO achieved state-of-the-art (SOTA) results across several key metrics. On the LLVIP dataset, it reached a Peak Signal-to-Noise Ratio (PSNR) of 63.258 dB and a Correlation Coefficient (CC) of 0.724, significantly outperforming recent models like DIVFusion and LENFusion.

Qualitatively, human perception tests indicated a strong preference for DFVO. Approximately 62.5% of volunteers preferred DFVO’s output over other SOTA methods. Participants specifically noted better color representation and a reduction in "whiting" (over-exposure) in areas with bright artificial lights, such as streetlamps.

Impact on Downstream Tasks

Beyond visual quality, the utility of DFVO was tested using a YOLOv5 object detector. When processing images fused by DFVO, the detector achieved 0.961 Precision and 0.758 Recall for pedestrian detection. This is a substantial improvement over using visible images alone (0.849 Precision / 0.682 Recall), proving that the "all-at-once" approach provides superior data for high-level machine vision.

Conclusion and Future Work

DFVO demonstrates that integrating enhancement and fusion into a single cascaded multi-task learning framework effectively mitigates the information loss inherent in traditional pipelines. While the model shows exceptional performance, the authors note that the spatial complexity of the Cross-Attention Module ($O(n^2)$) and the parameter count (9.57 million) present challenges for deployment on resource-constrained hardware. Future research will focus on developing a more lightweight feature extraction model to maintain high contrast and detail while reducing the computational footprint for real-time autonomous applications.

Summary updated

BibTeX
@article{zhou2025dfvo,
  author = {Qi Zhou and Yukai Shi and Xiaojun Yang and Xiaoyu Xian and Lunjia Liao and Ruimao Zhang and Liang Lin},
  title = {{DFVO: Learning Darkness-Free Visible and Infrared Image Disentanglement and Fusion All at Once}},
  journal = {IEEE Trans. Instrumentation \& Measurement},
  year = {2025},
  doi = {10.1109/tim.2025.3572155},
}
Figure