Abstract
Distance Metric Learning (DML) seeks to learn a discriminative embedding where similar examples are closer, and dissimilar examples are apart. In this paper, we address the problem of Semi-Supervised DML (SSDML) that tries to learn a metric using a few labeled examples, and abundantly available unlabeled examples. SSDML is important because it is infeasible to manually annotate all the examples present in a large dataset. Surprisingly, with the exception of a few classical approaches that learn a linear Mahalanobis metric, SSDML has not been studied in the recent years, and lacks approaches in the deep SSDML scenario. In this paper, we address this challenging problem, and revamp SSDML with respect to deep learning. In particular, we propose a stochastic, graph-based approach that first propagates the affinities between the pairs of examples from labeled data, to that of the unlabeled pairs. The propagated affinities are used to mine triplet based constraints for metric learning. We impose orthogonality constraint on the metric parameters, as it leads to a better performance by avoiding a model collapse.
Introduction
Distance Metric Learning (DML) is fundamental to many computer vision tasks, aiming to map data into a feature space where semantic similarity corresponds to geometric proximity. While supervised DML has achieved significant success, its reliance on exhaustive labeling makes it impractical for large-scale real-world applications. Semi-supervised learning (SSL) offers a promising alternative by leveraging a small set of labeled data alongside a vast amount of unlabeled data. However, SSDML research has largely stagnated, remaining focused on classical linear Mahalanobis metrics.
This work "resurrects" SSDML for the deep learning era. We propose a framework that enables neural networks to learn powerful discriminative embeddings using only a tiny fraction of labeled data. By combining graph-based affinity propagation with an orthogonality-constrained angular triplet loss, we bridge the gap between classical semi-supervised techniques and modern deep architectures.
Proposed Methodology
The proposed framework consists of four primary components designed to effectively utilize unlabeled data for metric learning.
1. Affinity Propagation
The process begins by constructing a k-Nearest Neighbor (kNN) graph of the entire dataset. Similarity information is then "spread" from the sparse labeled pairs to the unlabeled pairs using a random-walk-based propagation technique. This allows the model to estimate pairwise relationships in the unlabeled pool based on the manifold structure of the data.
2. Stochastic Triplet Mining
Using the propagated affinities, the model identifies "anchors," "positives," and "negatives" from the unlabeled data. An unlabeled point with high propagated affinity to an anchor is treated as a positive, while low affinity indicates a negative. This probabilistic approach to "pseudo-labeling" is more resilient to noise than hard assignments.
3. Angular Triplet Loss
To optimize the embedding, we employ an angular variant of the triplet loss. Unlike standard Euclidean distance losses, the angular loss captures third-order relationships between points and is inherently more robust to scaling issues, facilitating more stable training.
4. Orthogonality Constraint
A critical technical contribution is the imposition of an orthogonality constraint on the weights of the final embedding layer ($L^T L = I$). This constraint prevents "model collapse"—a common failure mode in semi-supervised learning where the network maps all points to a single coordinate to trivially minimize the loss. By invoking the Johnson-Lindenstrauss Lemma, we demonstrate that an orthogonal projection preserves the relative distances of the original data, maintaining the underlying structure of the unlabeled set.
Experimental Evaluation
We evaluated the proposed method across five standard benchmarks, including general object recognition (MNIST, Fashion-MNIST, CIFAR-10) and fine-grained categorization (CUB-200, Cars-196).
Quantitative Results
The proposed method consistently outperformed deep extensions of classical SSDML algorithms (e.g., LRML, SERAPH) and unsupervised methods (e.g., DeepCluster).
- MNIST: Achieved a Recall@1 of 93.9% with minimal labels, significantly higher than the 88.7% achieved by the best semi-supervised baseline.
- Cars-196: Reached a Recall@1 of 45.7%, a substantial improvement over the best unsupervised competitor (DeepCluster at 32.6%).
- Ablation Study: Removing the orthogonality constraint on CIFAR-10 led to a drop in NMI from 25.3% to 19.0%, confirming its necessity in preventing degenerate embeddings.
Discussion and Conclusion
This work successfully revives SSDML by integrating graph-based affinity propagation with deep neural networks. The introduction of orthogonality constraints proves vital for stabilizing semi-supervised training and preventing model collapse.
While effective, the method's performance is somewhat sensitive to the quality of the initial kNN graph. Future work could explore iterative graph refinement or more efficient propagation techniques for billion-scale datasets. Overall, this framework provides a robust foundation for treating metric learning as a semi-supervised problem, reducing the burden of manual annotation in deep learning.
Summary updated