Explicit Domain Adaptation with Loosely Coupled Samples

Oliver Scheel, Loren Arthur Schwarz, Nassir Navab, Federico Tombari

Abstract

Transfer learning is essential for deploying autonomous systems across diverse environments, yet traditional methods often suffer from a lack of interpretability and high data requirements. This paper introduces a framework for explicit domain adaptation using "loosely coupled" samples. By prepending a lightweight "Converter" module to a pre-trained model, the system learns an inspectable transformation matrix that maps target domain data back to the source domain. Evaluated on tasks ranging from rotated MNIST to real-world lane-change prediction, the proposed method demonstrates significant improvements in data efficiency and safety-critical explainability.

Introduction

In the field of autonomous driving, a model trained on sunny German highways may fail when deployed on rainy US city streets due to the "domain gap." While transfer learning techniques like fine-tuning or Generative Adversarial Networks (GANs) are commonly used to bridge this gap, they present three major challenges:

  1. Black-box Nature: Verifying how a model has adapted is difficult, posing a significant safety risk.
  2. Data Hunger: Fine-tuning typically requires extensive labeled datasets in the new domain.
  3. Hallucination: Generative methods may introduce non-existent artifacts into the data.

This research addresses these issues by proposing an explicit, inspectable transformation process that requires very few samples from the target domain.

Methodology

The core innovation is the Converter (C), a small neural network integrated before a pre-trained "base" model.

Explicit Transformation

Instead of processing input directly, the Converter calculates a transformation matrix ($T_x$) for each input. This matrix represents physical transformations—such as rotation, scaling, or translation—mapping the input back into the source domain. This explicit nature allows engineers to inspect the transformation to ensure its physical validity, a crucial feature for safety-critical applications.

Loosely Coupled Samples

Unlike standard supervised learning, which requires exact pairs (e.g., identical scenes captured by different sensors), this method utilizes "loose" correspondences. For instance, a "follow-lane" sequence in the source domain is paired with multiple different "follow-lane" sequences in the target domain.

Training Pipeline

The framework employs a three-step training process:

  1. Pre-training: The Converter is initialized with an "educated guess," such as an identity matrix.
  2. Correspondence Training: The Converter is trained using a novel loss function to minimize the distance between converted target samples and their loosely coupled source counterparts.
  3. Fine-tuning: A final light training pass is performed on the last layers of the main model to handle any remaining domain residuals.
The three-step training pipeline for the Converter module, illustrating pre-training, correspondence training, and fine-tuning.

Experimental Evaluation

The framework was tested across three distinct datasets:

  • Transformed MNIST: Standard digits (Domain A) vs. digits rotated by 180 degrees (Domain B).
  • Toy Sequences: Simulated lane-change trajectories with significant noise in Domain B.
  • Real-World Lane Change Prediction: Transferring knowledge from a proprietary BMW dataset (German highways) to the public NGSIM dataset (US highways). This task is particularly challenging due to differences in sensor setups, sampling frequencies, and driving behaviors.

Key Results

The proposed method (Ours-T2) outperformed baselines across all metrics, particularly in low-data regimes. On rotated MNIST with only 100 samples (0.14% of the dataset), the framework achieved 91.2% accuracy, significantly higher than standard fine-tuning (73.8%).

In real-world lane-change prediction on the NGSIM dataset, the model substantially reduced false positives (the "Frequency" metric). As shown in the table below, the framework achieved a total score of 0.691, nearly doubling the performance of baseline fine-tuning (0.424).

Performance metrics for lane-change prediction, showing the superiority of the proposed T1 and T2 models over fine-tuning across various sample sizes (b).

Discussion and Insights

The framework's primary advantage is interpretability as a safety feature. By forcing domain adaptation into an explicit matrix multiplication, the model's decisions become explainable. If a Converter applies an invalid transformation, it can be detected by human oversight. Furthermore, the framework allows for the integration of prior knowledge; for example, if a camera's mounting angle is known to have changed, the transformation matrix can be initialized accordingly to speed up convergence.

In lane-change experiments, the Converter effectively functioned as a learned filter, smoothing noisy US trajectories to match the clean German data the base model was trained on.

Conclusion

While the current approach relies on linear transformation matrices, it offers a powerful tool for efficient and transparent transfer learning. Future work will explore extending this to more complex sensor-to-sensor translations, such as mapping Radar data to Lidar-like representations, where explicit transformations provide immense value for verification.

Summary updated

Abstract

Transfer learning is an important field of machine learning in general, and particularly in the context of fully autonomous driving, which needs to be solved simultaneously for many different domains, such as changing weather conditions and country-specific driving behaviors. Traditional transfer learning methods often focus on image data and are black-box models. In this work we propose a transfer learning framework, core of which is learning an explicit mapping between domains. Due to its interpretability, this is beneficial for safety-critical applications, like autonomous driving. We show its general applicability by considering image classification problems and then move on to time-series data, particularly predicting lane changes. In our evaluation we adapt a pre-trained model to a dataset exhibiting different driving and sensory characteristics.
BibTeX
@article{scheel2020explicit,
  author = {Oliver Scheel and Loren Arthur Schwarz and Nassir Navab and Federico Tombari},
  title = {{Explicit Domain Adaptation with Loosely Coupled Samples}},
  journal = {IEEE Robotics Autom. Lett.},
  year = {2020},
  doi = {10.1109/lra.2020.3012127},
}
Figure