Abstract
This paper presents a novel multi-level metric learning framework designed to overcome the limitations of traditional distance metrics in handling misaligned and heterogeneous data. By leveraging a smoothed Wasserstein distance parameterized by both global and class-specific Mahalanobis matrices, the proposed method effectively aligns complex features while capturing fine-grained local idiosyncrasies. Experimental evaluations across person re-identification, facial kinship verification, and video classification tasks demonstrate that this approach significantly outperforms state-of-the-art baselines, particularly in scenarios involving significant geometric deformation and noise.
The Challenge: Misalignment and Heterogeneity
Metric learning typically aims to find an optimal Mahalanobis distance function that minimizes the distance between similar data points while maximizing it for dissimilar ones. However, real-world computer vision applications face two primary hurdles:
- Misaligned Data: Standard point-to-point distance metrics assume feature descriptors are perfectly aligned. In practice, factors like lighting changes, geometric deformations, or varying bounding box positions (e.g., a person appearing on different sides of a frame) lead to misalignment, causing traditional metrics to fail.
- Heterogeneous Data: Most existing methods learn a single global distance matrix. This "one-size-fits-all" approach often lacks the discriminative power needed to capture the unique, local characteristics of different classes within highly diverse datasets.
Proposed Method: Multi-Level Smoothed Wasserstein Metric
To address these challenges, the authors introduce a Multi-Level Metric Learning Model using a smoothed version of the Wasserstein distance (also known as Earth Mover's Distance).
Smoothed Wasserstein Distance
Unlike Euclidean distance, the Wasserstein distance calculates an optimal transport plan to "move" features from one sample to match another, naturally accounting for misalignment. To ensure computational efficiency and differentiability for gradient-based optimization, the authors employ an entropic smoothing term.
Multi-Level Distance Matrices
The core innovation lies in the parameterization of the "ground cost" within the Wasserstein distance. Instead of a single matrix, the model learns:
- A Global Matrix ($M_0$): Captures commonalities across all classes.
- Local Auxiliary Matrices ($M_t$): Capture class-specific idiosyncrasies.
The resulting ground distance for a specific class $t$ is defined by the combination $(M_0 + M_t)$. This dual-level structure allows the model to maintain universal feature importance while adapting to fine-grained details unique to specific categories.
Experimental Evaluation
The framework was evaluated across three distinct computer vision domains:
- Person Re-identification: On the PRID 2011 and iLIDS-VID datasets, the method achieved massive improvements. For instance, on PRID 2011, it reached a Rank-1 accuracy of 43.80%, compared to the 16.94% achieved by the LFDA baseline.
- Facial Kinship Verification: Tested on the KinFaceW-II dataset, the method showed top-tier performance, particularly in Mother-Son and Mother-Daughter relations, outperforming established methods like KISSME and LDML.
- Video Classification: On a Seattle highway traffic video database, the method achieved 91.36% accuracy, surpassing standard SVM and traditional LMNN variants.
Key Insights and Conclusion
By embedding a learnable Mahalanobis distance inside the Wasserstein ground cost, the model simultaneously learns how to align data (the transport plan) and what features are most discriminative (the metric matrix). This synergy, combined with the global-local matrix architecture, provides a robust solution for noisy, unaligned datasets. While computationally more intensive than global metrics due to the iterative nature of optimal transport calculations, the framework is highly generalizable and offers a powerful new tool for complex metric learning tasks.
Summary updated