Abstract
Computer vision systems typically struggle in extreme low-light environments where short-exposure images are dominated by noise and long-exposure images suffer from motion blur. While specialized models exist for image enhancement and object detection separately, combining them often requires massive annotated datasets of RAW images, which are expensive to produce. This paper introduces "YOLO in the Dark," a domain adaptation method that merges pre-trained expert models—a low-light enhancer and an object detector—using specialized "glue layers." By leveraging knowledge distillation to create a generative model, the authors train these layers to detect objects in RAW images using only standard RGB datasets. The resulting system achieves significant accuracy gains in near-total darkness while reducing computational costs by nearly 60% compared to traditional sequential pipelines.
Introduction
Standard computer vision pipelines are designed for well-lit RGB environments. In low-light scenarios, the signal-to-noise ratio drops significantly, rendering standard detectors like YOLO ineffective. While models such as "Learning-to-See-in-the-Dark" (SID) can reconstruct clear images from RAW sensor data, simply stacking an enhancer and a detector (the "naive" approach) is computationally expensive and often redundant.
The primary bottleneck in developing end-to-end low-light detectors is data: pairing RAW sensor data with object-level annotations is a labor-intensive process. The authors address this by proposing a method to "glue" existing expert models together, allowing a detector trained on standard RGB data (like COCO) to function effectively on the latent features produced by a RAW image encoder.
Methodology: Merging Experts via Glue Layers
The core of the "YOLO in the Dark" architecture is the transition from pixel-space processing to latent-feature-space merging. The process involves three primary components:
- Glue Layers: Instead of processing a fully reconstructed image, the authors extract fragments of the SID encoder and the YOLO detector. A "glue layer" is inserted between them to transform the latent features of the SID encoder into the specific format required by the YOLO backbone.
- Generative Model ($G2e$): To train the glue layer without new RAW labels, the authors developed a student model designed to be the inverse of the SID decoder. This model takes standard RGB images and generates "fake" latent features that mimic the output of a SID encoder processing a RAW image.
- Latent Feature Training: Using the COCO and PASCAL VOC datasets, RGB images are passed through the $G2e$ model to produce synthetic RAW features. These are then used to train the glue layer to map these features to the original YOLO latent space, effectively teaching the model to "see" RAW data using only RGB labels.
A critical insight during development was the importance of frequency preservation. The authors found that high-level abstract features alone were insufficient for accurate detection. To maintain the sharp edges and shapes necessary for recognition, they concatenated features from multiple scales (ranging from 1/1 to 1/8) within the glue layer architecture.
Experimental Results
The proposed method was evaluated using the See-in-the-Dark (SID) dataset for RAW low-light testing and standard datasets like COCO for training.
Detection Performance
"YOLO in the Dark" achieved a mean Average Precision (mAP) of 0.55, representing a 2.1x improvement over a naive YOLO model (0.26) applied to enhanced images. The system demonstrated extreme sensitivity, successfully detecting objects in environments with illuminance as low as 0.013 lux. This capability effectively reduces the required exposure time by 60% compared to standard methods.
Computational Efficiency
By bypassing the heavy SID decoder—which is responsible for reconstructing the image for human viewing—the model saves significant processing power. The computational load was reduced from 270.79 GMACs (for a sequential SID + YOLO pipeline) to 113.90 GMACs, making it much more suitable for real-time edge applications.
Conclusion and Future Work
"YOLO in the Dark" demonstrates that merging expert models in the latent feature space is a viable and efficient alternative to end-to-end training on specialized datasets. While the model currently trails slightly behind the absolute accuracy of the full sequential SID+YOLO approach, the massive reduction in computational overhead makes it a compelling solution for low-power vision systems. Future research will focus on refining glue layer optimization to close the accuracy gap and exploring the application of this domain adaptation method to other modalities, such as infrared or thermal imaging.
Summary updated