Abstract
Deep Metric Learning (DML) aims to learn a mapping where semantically similar items are clustered together. However, performance is often constrained by the limited diversity of training data. We present Spherical Embedding Expansion (SEE), a lightweight, "plug-and-play" augmentation technique that synthesizes new feature vectors directly in the embedding space. By leveraging Max-Mahalanobis distribution (MMD) centers within the nullspace of class proxies, SEE forces the model to learn more robust class boundaries. Our method requires no additional parameters and achieves state-of-the-art results across various benchmarks and architectures, including CNNs and Vision Transformers.
Introduction
Deep Metric Learning is foundational to modern computer vision tasks such as image retrieval and face recognition. While significant progress has been made in loss function design and hard-sample mining, these approaches are inherently limited by the semantic variations present in the training set. Previous attempts to synthesize training data using generative models (e.g., GANs) often introduce heavy computational overhead and optimization challenges.
SEE addresses these limitations by performing augmentation directly in the latent embedding space. This approach is computationally efficient and mathematically rigorous, ensuring that synthetic samples maintain geometric consistency with real data while expanding the explored territory of the hypersphere.
Methodology
The core of SEE is the generation of synthetic embeddings that challenge the model's current class boundaries.
Nullspace Exploration and MMD Sampling
For an embedding vector $z$ and its corresponding class proxy, SEE identifies a nullspace—a lower-dimensional sphere where points maintain the same distance from the proxy as the original sample. This ensures that synthetic points are "equally difficult" for the model to classify.
To determine the placement of these synthetic points, SEE utilizes Max-Mahalanobis distribution (MMD) centers. This strategy maximizes the angular dispersion between synthetic samples, ensuring the model explores the latent space as efficiently as possible.
Curriculum Learning and Integration
SEE employs a curriculum strategy, prioritizing augmentation for the "hardest" samples in a mini-batch—those most likely to be misclassified. The intensity of augmentation increases as training progresses. Because SEE operates at the output of the feature extractor, it can be seamlessly integrated with various loss functions like NSoftmax, ProxyNCA++, or ArcFace without modifying the underlying network architecture.
Experimental Results
We evaluated SEE on four standard benchmarks: CUB-200-2011, Cars-196, Stanford Online Product (SOP), and In-shop Clothes Retrieval.
As shown in the results table, SEE consistently provides a performance lift across all datasets and backbones. For instance, when paired with a ViT-S backbone on the Cars-196 dataset, SEE achieved a Recall@1 (R@1) of 88.8%, a significant improvement over the baseline. The method is particularly effective in low-dimensional embedding spaces (e.g., 128 dimensions), where it helps mitigate the "crowding" problem by forcing tighter, more distinct clusters.
Conclusion
SEE offers a robust and efficient solution for data scarcity in Deep Metric Learning. By synthesizing samples in the embedding space's nullspace using MMD centers, it enhances generalization without the overhead of traditional generative models. Future work may extend this spherical expansion logic to related fields such as out-of-distribution detection and long-tailed learning.
Summary updated