Cross-Domain Car Detection Using Unsupervised Image-to-Image Translation: From Day to Night

Vinicius F. Arruda, Thiago M. Paixão, Rodrigo Ferreira Berriel, Alberto F. De Souza, Claudine Badue, Nicu Sebe, Thiago Oliveira-SantosIJCNN 2019 · DOI · OpenReview

Abstract

Deep learning techniques have enabled the emergence of state-of-the-art models to address object detection tasks. However, these techniques are data-driven, delegating the accuracy to the training dataset which must resemble the images in the target task. The acquisition of a dataset involves annotating images, an arduous and expensive process, generally requiring time and manual effort. Thus, a challenging scenario arises when the target domain of application has no annotated dataset available, making tasks in such situation to lean on a training dataset of a different domain. Sharing this issue, object detection is a vital task for autonomous vehicles where the large amount of driving scenarios yields several domains of application requiring annotated data for the training process. In this work, a method for training a car detection system with annotated data from a source domain (day images) without requiring the image annotations of the target domain (night images) is presented. For that, a model based on Generative Adversarial Networks (GANs) is explored to enable the generation of an artificial dataset with its respective annotations. The artificial dataset (fake dataset) is created translating images from day-time domain to night-time domain. The fake dataset, which comprises annotated images of only the target domain (night images), is then used to train the car detector model. Experimental results showed that the proposed method achieved significant and consistent improvements, including the increasing by more than 10% of the detection performance when compared to the training with only the available annotated data (i.e., day images).

Introduction

Object detection models are notoriously data-hungry and sensitive to the specific domain on which they were trained. A car detector optimized for sunny, daytime images often fails significantly when deployed at night due to the "domain gap"—the stark differences in lighting, glare, and shadow patterns.

Traditionally, the solution to this gap is to collect and manually annotate thousands of nighttime images. However, manual annotation is expensive, slow, and tedious. This research addresses a critical question for autonomous vehicle systems: How can we train a high-performing nighttime car detector using only annotated daytime data and unannotated nighttime footage?

Methodology

The proposed approach utilizes Unsupervised Domain Adaptation (UDA) to bridge the gap between daytime and nighttime imagery. The pipeline consists of three primary stages:

  1. Artificial Dataset Generation: The researchers employ CycleGAN, a generative adversarial network capable of "unpaired" image-to-image translation. Unlike traditional supervised methods, CycleGAN does not require identical "before and after" shots of the same scene. Instead, it learns the general stylistic features of the "night" domain from a collection of random night photos and applies those characteristics to daytime photos.
  2. Annotation Transfer: A key advantage of CycleGAN is its ability to preserve the structural consistency (content) of an image while altering its style. Because the geometry of the scene remains constant, the authors can automatically transfer bounding box coordinates from the original daytime images to the newly generated "fake night" images.
  3. Detector Training: Using the Faster R-CNN architecture, the authors trained the detector on various combinations of datasets—original day images, fake night images, and mixed sets—to identify the most robust configuration for real-world nighttime inference.
Overview of the proposed pipeline for artificial dataset generation and car detector training.

Experimental Setup and Datasets

The study utilized the Berkeley Deep Drive (BDD) dataset, a comprehensive collection of 1280x720 driving videos. To ensure high-quality training, the authors filtered the dataset for clear or partly cloudy weather across highway, city, and residential scenes. To manage the computational demands of GAN training, images were preprocessed by cropping and rescaling to a resolution of 256 x 256 pixels. A total of 12,000 images were used, split between training and testing sets for both domains.

Key Results

The performance of the detectors was evaluated using mean Average Precision (mAP) on real nighttime images. The results demonstrate the efficacy of the translation-based approach:

  • Baseline (Day-only training): 74.2% mAP. This represents the performance floor, highlighting the severe impact of the domain gap.
  • Proposed Method (Fake Night-only training): 84.7% mAP. This represents a 10.5% improvement achieved entirely through automated translation without a single new manual label.
  • Mixed Training (Day + Fake Night): 86.6% mAP. Combining original data with translated data provided the best results, acting as a form of data augmentation that helps the model learn features invariant to lighting conditions.
  • Upper Bound (Real Night training): 92.0% mAP. While the unsupervised method does not fully close the gap to a manually annotated nighttime dataset, it recovers more than half of the performance lost to the domain gap.
Performance comparison (mAP) of the car detector trained on different datasets and tested on real nighttime images.

Discussion and Future Work

One notable insight from the study is that the detector was remarkably resilient to GAN-generated artifacts. Even when the CycleGAN produced non-photorealistic elements (such as glowing clouds), the detector successfully focused on the structural features of the vehicles.

However, limitations remain. The 256x256 resolution constraint likely hindered the detection of small or distant vehicles. Future work could explore higher-resolution GAN architectures and extend the methodology to more complex classes, such as pedestrians or cyclists, where structural preservation during translation is even more critical for safety-critical autonomous driving applications.

Summary updated

Abstract

Deep learning techniques have enabled the emergence of state-of-the-art models to address object detection tasks. However, these techniques are data-driven, delegating the accuracy to the training dataset which must resemble the images in the target task. The acquisition of a dataset involves annotating images, an arduous and expensive process, generally requiring time and manual effort. Thus, a challenging scenario arises when the target domain of application has no annotated dataset available, making tasks in such situation to lean on a training dataset of a different domain. Sharing this issue, object detection is a vital task for autonomous vehicles where the large amount of driving scenarios yields several domains of application requiring annotated data for the training process. In this work, a method for training a car detection system with annotated data from a source domain (day images) without requiring the image annotations of the target domain (night images) is presented. For that, a model based on Generative Adversarial Networks (GANs) is explored to enable the generation of an artificial dataset with its respective annotations. The artificial dataset (fake dataset) is created translating images from day-time domain to night-time domain. The fake dataset, which comprises annotated images of only the target domain (night images), is then used to train the car detector model. Experimental results showed that the proposed method achieved significant and consistent improvements, including the increasing by more than 10% of the detection performance when compared to the training with only the available annotated data (i.e., day images).
BibTeX
@inproceedings{arruda2019cross,
  author = {Vinicius F. Arruda and Thiago M. Paixão and Rodrigo Ferreira Berriel and Alberto F. De Souza and Claudine Badue and Nicu Sebe and Thiago Oliveira-Santos},
  title = {{Cross-Domain Car Detection Using Unsupervised Image-to-Image Translation: From Day to Night}},
  booktitle = {IJCNN},
  year = {2019},
  doi = {10.1109/ijcnn.2019.8852008},
}
Figure