Abstract
Domain Adaptation (DA) remains a significant challenge in machine learning, where models trained on source data often fail on target data due to distribution shifts. This paper introduces MLOT (Metric Learning in Optimal Transport), a novel framework that improves unsupervised domain adaptation by jointly learning a custom Mahalanobis distance metric and a transportation plan. A key theoretical contribution is the proof that Principal Component Analysis (PCA) is the optimal linear projection for minimizing Wasserstein distances, providing a rigorous foundation for the method. MLOT consistently outperforms standard Optimal Transport techniques on computer vision benchmarks.
Introduction
The performance of machine learning models is frequently compromised when the deployment environment (target domain) differs from the training environment (source domain). Unsupervised Domain Adaptation (UDA) seeks to mitigate this "domain shift" without requiring labels for the target data. Optimal Transport (OT) has emerged as a powerful tool for UDA, treating data distributions as masses to be moved efficiently. However, standard OT relies on fixed metrics like Euclidean distance, which may not align with the intrinsic geometry of the data shift.
Methodology: Metric Learning in Optimal Transport
The proposed MLOT algorithm addresses the limitations of fixed-metric OT by jointly optimizing dimensionality reduction, the transportation plan, and the ground metric.
PCA as a Wasserstein Optimizer
A significant theoretical insight of this work is the connection between classical dimensionality reduction and modern transport theory. The authors prove that PCA is the optimal linear projection for minimizing the Wasserstein distance between a high-dimensional distribution and its lower-dimensional representation. This result justifies the use of PCA as a starting point for domain alignment.
The MLOT Algorithm
MLOT operates through an iterative process that alternates between two primary steps:
- Optimal Transport: Given the current metric, the Sinkhorn-Knopp algorithm is used to find the most efficient transportation plan between domains.
- Metric Learning: The source domain's metric is updated via gradient descent to minimize alignment error, guided by a new generalization bound derived by the authors.
Experimental Evaluation
The authors evaluated MLOT on standard computer vision benchmarks, including Office-Caltech and Office31, using both traditional SURF features and modern DeCAF deep learning features.
Key Results
- SURF Features: On the Office-Caltech dataset, MLOT achieved an average accuracy of 49.7%, significantly outperforming standard OTDA (48.2%) and the "No Adaptation" baseline (31.4%).
- Deep Features: Utilizing DeCAF6 features, MLOT reached 84.7% accuracy, representing a 1.5-point improvement over the next best non-deep-learning method.
- Robustness: The method demonstrated superior performance in difficult cross-domain scenarios, such as adapting from Webcam images to Amazon product images.
Discussion and Conclusion
MLOT demonstrates that learning the ground metric is essential for effective Optimal Transport in domain adaptation. By integrating metric learning with dimensionality reduction and transportation planning, the method provides a theoretically grounded and empirically successful approach to UDA. While currently focused on linear Mahalanobis distances, future work may extend this framework to non-linear shifts and end-to-end deep learning architectures.
Summary updated