Abstract
Deep learning-based object detection models often suffer from significant performance degradation when encountering "domain shift"—the discrepancy between training and testing environments (e.g., transitioning from day to night). This paper investigates a novel 6-channel image representation designed to bridge this gap. By concatenating original RGB images with their GAN-generated counterparts from a target domain, the authors aim to provide detectors with a richer, cross-domain feature set. While the 6-channel approach improves upon source-only baselines, experimental results on the BDD100K dataset suggest that the noise introduced by image-to-image translation can hinder the model's ability to outperform simpler data augmentation strategies.
Introduction
Object detection is a cornerstone of autonomous systems, yet models remain notoriously sensitive to environmental changes. A system trained exclusively on daytime footage often fails in low-light conditions, rain, or fog. While collecting and labeling massive datasets for every possible domain is prohibitively expensive, Generative Adversarial Networks (GANs) offer a potential solution by "translating" existing data into target domains.
Most existing methods use GANs for standard data augmentation—adding "fake" images to the training pool. This paper proposes a different strategy: representation augmentation. Instead of choosing between a real image and its translated counterpart, the proposed method utilizes both simultaneously in a unified 6-channel input format.
Methodology
The proposed pipeline expands the standard 3-channel (RGB) input into a 6-channel representation through a multi-step process:
- Image-to-Image Translation: The authors employ CycleGAN, an unsupervised model capable of learning translations between domains (e.g., Day $\leftrightarrow$ Night) without paired examples.
- 6-Channel Concatenation: For every image, a counterpart in the opposite domain is generated. The original 3-channel RGB image is then concatenated with the 3-channel GAN-generated image to form a single 6-channel input.
- Architectural Modification: The study utilizes a Faster R-CNN with a ResNet-101 backbone. The primary modification is to the first convolutional layer, which is adjusted to accept 6 input channels instead of 3.
This 6-channel format is maintained during both training and inference. For instance, when testing on a real night image, the system first generates a "fake day" version and stacks it with the original before processing.
Experimental Setup
The study utilizes the BDD100K (Berkeley DeepDrive) dataset, focusing on a subset of 12,000 images split between clear day and night scenes. The primary task is the detection of cars, with performance measured by mean Average Precision (mAP).
Results and Discussion
The authors compared the 6-channel approach against several baselines when testing on night scenes:
| Training Configuration | mAP (Night Test) |
|---|---|
| Baseline (Real Day only) | 0.777 |
| Proposed 6-Channel (Day + Fake Night) | 0.830 |
| Standard GAN (Fake Night only) | 0.893 |
| Upper Bound (Real Night only) | 0.933 |
While the 6-channel representation significantly outperformed the day-only baseline, it surprisingly underperformed compared to the standard 3-channel model trained solely on GAN-generated "fake night" images.
This discrepancy highlights the "Quality Shift" trap. GAN-generated images often contain artifacts or noise that differ from real-world image statistics. When these are concatenated with real images, the detector may struggle to extract useful features from the noisy channels, potentially treating them as interference rather than helpful context. The results suggest that the raw 3-channel "fake" image provides a stronger, more focused signal for the detector than the combined 6-channel stack.
Conclusion
This research demonstrates that while 6-channel representations can improve cross-domain robustness, they are currently limited by the quality of image-to-image translation. Future work will focus on improving translation fidelity to reduce noise and exploring more sophisticated fusion techniques—such as attention mechanisms—rather than simple concatenation to better integrate multi-domain information.
Summary updated