Abstract
Metric learning is a fundamental task in distance-based classification, yet traditional methods often struggle with noisy or mislabeled training data. This paper introduces Robust Geometric Metric Learning (RGML), a framework that reframes metric learning as a robust covariance matrix estimation problem on Riemannian manifolds. By jointly estimating class-specific covariance matrices and their Riemannian barycenter, RGML achieves high classification accuracy and significant resilience against label noise.
Introduction
Distance-based classifiers, such as K-Nearest Neighbors (k-NN), depend heavily on the choice of distance metric. While Euclidean distance is standard, it often fails to account for the underlying distribution of data within classes. Metric learning addresses this by learning a Mahalanobis distance—a linear transformation defined by a Symmetric Positive Definite (SPD) matrix—that optimizes class separation.
However, many existing fast metric learning algorithms, such as Geometric Mean Metric Learning (GMML), rely on Sample Covariance Matrices (SCM). SCM is notoriously sensitive to outliers and mislabeled data, causing performance to degrade rapidly in real-world, noisy scenarios.
Methodology
The proposed RGML framework jointly estimates regularized covariance matrices for each class while shrinking them toward a common Riemannian barycenter. Unlike Euclidean-based methods, RGML operates on the Riemannian manifold of SPD matrices.
Two primary variants are proposed:
- RGML Gaussian: Assumes Gaussian data distribution and minimizes negative log-likelihood.
- RGML Tyler: Utilizes Tyler’s M-estimator for robustness against heavy-tailed distributions and outliers. This variant operates on a submanifold of matrices with unit determinants to maintain scale invariance.
Optimization is performed using Riemannian gradient descent. The authors prove that the cost functions are geodesically convex (g-convex), ensuring that any local minimum found is a global minimum.
Experimental Results
The framework was evaluated on the Wine, Vehicle, and Iris datasets from the UCI repository. To test robustness, label noise was introduced by randomly flipping 5%, 10%, and 15% of training labels.
Performance and Robustness
On clean data, RGML Gaussian consistently outperformed baselines like ITML, GMML, and LMNN. As label noise increased, RGML Tyler demonstrated superior resilience. For instance, on the Wine dataset with 10% mislabeling, RGML Tyler maintained an error rate of 4.51%, significantly lower than SCM (13.70%) and GMML (5.71%).
Efficiency
The Riemannian gradient descent approach is computationally efficient. As shown in the convergence plots, the algorithms typically reach a stable solution in fewer than 20 iterations, demonstrating the practical utility of the g-convex optimization framework.
Discussion and Conclusion
RGML successfully bridges the gap between robust statistics and metric learning by leveraging Riemannian geometry. While the current approach is limited to linear Mahalanobis distances, its g-convexity provides strong theoretical guarantees for global optimality. Future work may extend this framework to non-linear kernels or other robust loss functions.
Summary updated