Six-channel Image Representation for Cross-domain Object Detection

Tianxiao Zhang, Wenchi Ma, Guanghui WangarXiv 2021 · arXiv

Abstract

Deep learning-based object detection models often suffer from significant performance degradation when encountering "domain shift"—the discrepancy between training and testing environments (e.g., transitioning from day to night). This paper investigates a novel 6-channel image representation designed to bridge this gap. By concatenating original RGB images with their GAN-generated counterparts from a target domain, the authors aim to provide detectors with a richer, cross-domain feature set. While the 6-channel approach improves upon source-only baselines, experimental results on the BDD100K dataset suggest that the noise introduced by image-to-image translation can hinder the model's ability to outperform simpler data augmentation strategies.

Introduction

Object detection is a cornerstone of autonomous systems, yet models remain notoriously sensitive to environmental changes. A system trained exclusively on daytime footage often fails in low-light conditions, rain, or fog. While collecting and labeling massive datasets for every possible domain is prohibitively expensive, Generative Adversarial Networks (GANs) offer a potential solution by "translating" existing data into target domains.

Most existing methods use GANs for standard data augmentation—adding "fake" images to the training pool. This paper proposes a different strategy: representation augmentation. Instead of choosing between a real image and its translated counterpart, the proposed method utilizes both simultaneously in a unified 6-channel input format.

Methodology

The proposed pipeline expands the standard 3-channel (RGB) input into a 6-channel representation through a multi-step process:

  1. Image-to-Image Translation: The authors employ CycleGAN, an unsupervised model capable of learning translations between domains (e.g., Day $\leftrightarrow$ Night) without paired examples.
  2. 6-Channel Concatenation: For every image, a counterpart in the opposite domain is generated. The original 3-channel RGB image is then concatenated with the 3-channel GAN-generated image to form a single 6-channel input.
  3. Architectural Modification: The study utilizes a Faster R-CNN with a ResNet-101 backbone. The primary modification is to the first convolutional layer, which is adjusted to accept 6 input channels instead of 3.
Overview of the 6-channel training and testing pipeline.

This 6-channel format is maintained during both training and inference. For instance, when testing on a real night image, the system first generates a "fake day" version and stacks it with the original before processing.

Experimental Setup

The study utilizes the BDD100K (Berkeley DeepDrive) dataset, focusing on a subset of 12,000 images split between clear day and night scenes. The primary task is the detection of cars, with performance measured by mean Average Precision (mAP).

Examples of real day images (top) and their CycleGAN-generated fake night counterparts (bottom) from the BDD100K dataset.

Results and Discussion

The authors compared the 6-channel approach against several baselines when testing on night scenes:

Training Configuration mAP (Night Test)
Baseline (Real Day only) 0.777
Proposed 6-Channel (Day + Fake Night) 0.830
Standard GAN (Fake Night only) 0.893
Upper Bound (Real Night only) 0.933

While the 6-channel representation significantly outperformed the day-only baseline, it surprisingly underperformed compared to the standard 3-channel model trained solely on GAN-generated "fake night" images.

This discrepancy highlights the "Quality Shift" trap. GAN-generated images often contain artifacts or noise that differ from real-world image statistics. When these are concatenated with real images, the detector may struggle to extract useful features from the noisy channels, potentially treating them as interference rather than helpful context. The results suggest that the raw 3-channel "fake" image provides a stronger, more focused signal for the detector than the combined 6-channel stack.

Conclusion

This research demonstrates that while 6-channel representations can improve cross-domain robustness, they are currently limited by the quality of image-to-image translation. Future work will focus on improving translation fidelity to reduce noise and exploring more sophisticated fusion techniques—such as attention mechanisms—rather than simple concatenation to better integrate multi-domain information.

Summary updated

BibTeX
@misc{zhang2021six,
  author = {Tianxiao Zhang and Wenchi Ma and Guanghui Wang},
  title = {{Six-channel Image Representation for Cross-domain Object Detection}},
  year = {2021},
  eprint = {2101.00561},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2101.00561},
}
Figure