An Optimal Transport Approach to Deep Metric Learning (Student Abstract)

Jason Xiaotian Dou, Lei Luo, Raymond Mingrui YangAAAI 2022 · DOI

Introduction

Deep Metric Learning (DML) is a fundamental task in computer vision, focused on learning a feature embedding space where semantically similar images are positioned close together and dissimilar ones are pushed apart. While Euclidean distance is the standard metric used in DML, it often fails to account for the complex, non-linear geometry of high-dimensional image manifolds. This research introduces a novel framework that leverages Wasserstein distance—a concept from Optimal Transport—to better capture these geometric relationships. By integrating Sinkhorn divergences and a specialized coreset selection algorithm, the authors achieve state-of-the-art performance while addressing the inherent computational challenges of Optimal Transport.

The Deep Wasserstein Metric Learning Framework

The core contribution of this work is the Deep Wasserstein Metric Learning framework, which transitions from Euclidean-based distance measures to those derived from Optimal Transport. The framework consists of three primary components:

1. Wasserstein-based Loss Functions

The authors redefine traditional ranking losses, such as Contrastive and Triplet losses, by substituting the Euclidean distance with the $W_1$ Wasserstein distance. These new formulations, termed wcontrastive and wtriplet, allow the model to measure the distance between image distributions rather than simple point-wise vectors, capturing more nuanced structural information.

2. Sinkhorn Divergence

Computing the exact Wasserstein distance is computationally prohibitive for deep learning training loops. To overcome this, the authors utilize Sinkhorn divergence, an entropically regularized version of Optimal Transport. This approximation is not only faster but also provides a smooth, differentiable objective that is well-suited for backpropagation and gradient descent.

3. Wasserstein Coresets (WCRAIG)

To further enhance efficiency, the paper introduces WCRAIG, a coreset selection method tailored for Wasserstein distances. By identifying a small, representative subset of the training data (a "coreset") that preserves the statistical properties of the full set, the authors significantly reduce the number of expensive distance computations required during training without sacrificing model accuracy.

Experimental Evaluation

The proposed framework was rigorously tested on two standard benchmarks for image retrieval and clustering:

  • CUB200-2011: A fine-grained dataset containing 200 bird species.
  • Stanford Online Products (SOP): A large-scale dataset with thousands of product categories, characterized by high intra-class variance.

Results and Analysis

The experimental results demonstrate that the Wasserstein-based approach consistently outperforms traditional DML methods across various metrics, including Recall@K (R@K), Normalized Mutual Information (NMI), F1 score, and mean Average Precision (mAP).

Stanford Online Products (SOP)

On the SOP dataset, the wtriplet loss achieved a Recall@1 of 0.7400, surpassing the previous state-of-the-art Multisimilarity loss (0.7399) and providing a massive improvement over the standard Triplet loss (0.6094).

Performance comparison on the Stanford Online Products (SOP) dataset.

CUB200-2011

The gains were even more pronounced on the CUB200-2011 dataset. The wtriplet method reached an R@1 of 0.7277, significantly outperforming Margin loss (0.6493) and the baseline Triplet loss (0.5949).

Performance comparison on the CUB200-2011 bird species dataset.

These results suggest that moving from batch-level to image-level Wasserstein measures allows the model to capture finer-grained relationships, leading to more coherent clusters and more accurate retrieval.

Conclusion and Future Work

This research demonstrates that Optimal Transport provides a powerful alternative to Euclidean distance for Deep Metric Learning. By utilizing Sinkhorn divergences and Wasserstein coresets, the authors make these theoretically robust measures practically viable. While the computational cost remains higher than Euclidean methods, the significant performance improvements on benchmark datasets justify the approach. Future research will likely focus on further optimizing the training efficiency to enable even larger-scale applications.

Summary updated

BibTeX
@inproceedings{dou2022optimal,
  author = {Jason Xiaotian Dou and Lei Luo and Raymond Mingrui Yang},
  title = {{An Optimal Transport Approach to Deep Metric Learning (Student Abstract)}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2022},
  doi = {10.1609/aaai.v36i11.21604},
}
Figure