Multi-Level Metric Learning via Smoothed Wasserstein Distance

Jie Xu, Lei Luo, Cheng Deng, Heng Huang

Abstract

This paper presents a novel multi-level metric learning framework designed to overcome the limitations of traditional distance metrics in handling misaligned and heterogeneous data. By leveraging a smoothed Wasserstein distance parameterized by both global and class-specific Mahalanobis matrices, the proposed method effectively aligns complex features while capturing fine-grained local idiosyncrasies. Experimental evaluations across person re-identification, facial kinship verification, and video classification tasks demonstrate that this approach significantly outperforms state-of-the-art baselines, particularly in scenarios involving significant geometric deformation and noise.

The Challenge: Misalignment and Heterogeneity

Metric learning typically aims to find an optimal Mahalanobis distance function that minimizes the distance between similar data points while maximizing it for dissimilar ones. However, real-world computer vision applications face two primary hurdles:

  1. Misaligned Data: Standard point-to-point distance metrics assume feature descriptors are perfectly aligned. In practice, factors like lighting changes, geometric deformations, or varying bounding box positions (e.g., a person appearing on different sides of a frame) lead to misalignment, causing traditional metrics to fail.
  2. Heterogeneous Data: Most existing methods learn a single global distance matrix. This "one-size-fits-all" approach often lacks the discriminative power needed to capture the unique, local characteristics of different classes within highly diverse datasets.
Examples of misaligned person re-identification data from multi-shot sequences, highlighting the difficulty of direct point-to-point comparison.

Proposed Method: Multi-Level Smoothed Wasserstein Metric

To address these challenges, the authors introduce a Multi-Level Metric Learning Model using a smoothed version of the Wasserstein distance (also known as Earth Mover's Distance).

Smoothed Wasserstein Distance

Unlike Euclidean distance, the Wasserstein distance calculates an optimal transport plan to "move" features from one sample to match another, naturally accounting for misalignment. To ensure computational efficiency and differentiability for gradient-based optimization, the authors employ an entropic smoothing term.

Multi-Level Distance Matrices

The core innovation lies in the parameterization of the "ground cost" within the Wasserstein distance. Instead of a single matrix, the model learns:

  • A Global Matrix ($M_0$): Captures commonalities across all classes.
  • Local Auxiliary Matrices ($M_t$): Capture class-specific idiosyncrasies.

The resulting ground distance for a specific class $t$ is defined by the combination $(M_0 + M_t)$. This dual-level structure allows the model to maintain universal feature importance while adapting to fine-grained details unique to specific categories.

Illustration of the metric learning objective: the model learns to pull similar samples (red circles) together while pushing dissimilar samples (colored squares) outside a defined margin.

Experimental Evaluation

The framework was evaluated across three distinct computer vision domains:

  • Person Re-identification: On the PRID 2011 and iLIDS-VID datasets, the method achieved massive improvements. For instance, on PRID 2011, it reached a Rank-1 accuracy of 43.80%, compared to the 16.94% achieved by the LFDA baseline.
  • Facial Kinship Verification: Tested on the KinFaceW-II dataset, the method showed top-tier performance, particularly in Mother-Son and Mother-Daughter relations, outperforming established methods like KISSME and LDML.
  • Video Classification: On a Seattle highway traffic video database, the method achieved 91.36% accuracy, surpassing standard SVM and traditional LMNN variants.

Key Insights and Conclusion

By embedding a learnable Mahalanobis distance inside the Wasserstein ground cost, the model simultaneously learns how to align data (the transport plan) and what features are most discriminative (the metric matrix). This synergy, combined with the global-local matrix architecture, provides a robust solution for noisy, unaligned datasets. While computationally more intensive than global metrics due to the iterative nature of optimal transport calculations, the framework is highly generalizable and offers a powerful new tool for complex metric learning tasks.

Summary updated

Abstract

Traditional metric learning methods aim to learn a single Mahalanobis distance metric M, which, however, is not discriminative enough to characterize the complex and heterogeneous data. Besides, if the descriptors of the data are not strictly aligned, Mahalanobis distance would fail to exploit the relations among them. To tackle these problems, in this paper, we propose a multi-level metric learning method using a smoothed Wasserstein distance to characterize the errors between any two samples, where the ground distance is considered as a Mahalanobis distance. Since smoothed Wasserstein distance provides not only a distance value but also a flow-network indicating how the probability mass is optimally transported between the bins, it is very effective in comparing two samples whether they are aligned or not. In addition, to make full use of the global and local structures that exist in data features, we further model the commonalities between various classification through a shared distance matrix and the classification-specific idiosyncrasies with additional auxiliary distance matrices. An efficient algorithm is developed to solve the proposed new model. Experimental evaluations on four standard databases show that our method obviously outperforms other state-of-the-art methods.
BibTeX
@inproceedings{xu2018multi,
  author = {Jie Xu and Lei Luo and Cheng Deng and Heng Huang},
  title = {{Multi-Level Metric Learning via Smoothed Wasserstein Distance}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2018},
  doi = {10.24963/ijcai.2018/405},
}
Figure