Abstract
Few-shot learning (FSL) remains a significant challenge in computer vision, primarily due to the risks of overfitting and limited transferability when training on sparse data. This paper presents RankDNN, a novel framework that reformulates the few-shot classification task as a binary relevance ranking problem. Instead of predicting class labels directly, RankDNN learns to determine which of two support images is more relevant to a given query. By employing a triplet-based training strategy and a vector-Kronecker product encoding scheme, the model captures high-order correlations that are more robust across different domains. Our evaluations on standard benchmarks like miniImageNet and CUB show that RankDNN not only achieves state-of-the-art accuracy but also demonstrates exceptional performance in cross-domain scenarios.
Introduction
The primary goal of few-shot learning is to enable models to generalize to new concepts from only a handful of examples. Traditional FSL methods typically treat this as a standard classification problem. However, this approach often suffers from two critical drawbacks. First, deep networks tend to overfit the limited support set, memorizing specific noise rather than general features. Second, the learned "meta-knowledge" is often tied to the training categories, leading to poor performance when the model is tested on a completely different domain (e.g., training on general objects and testing on fine-grained bird species).
RankDNN addresses these issues by shifting the focus from "what an object is" to "how to compare objects." By treating FSL as a ranking task, the model learns domain-agnostic similarity patterns that are more transferable.
Methodology
Task Reformulation
RankDNN transforms the standard $N$-way $K$-shot classification task into a binary rank order classification task. As shown in Figure 1, the model is presented with a query and multiple support classes. Instead of a direct multi-class prediction, the problem is broken down into comparing the query against pairs of support images to determine relative similarity.
The Triplet Ranking Approach
The core of RankDNN is the use of triplets. Each triplet consists of a query image ($x_q$), a positive support image ($x_i$ from the same class), and a negative support image ($x_j$ from a different class). The objective is to predict whether $x_i$ is more relevant to $x_q$ than $x_j$. This approach significantly increases the amount of available training data, as the number of possible triplets grows cubically with the number of samples.
Feature Encoding and RankMLP
To effectively compare images within a triplet, RankDNN utilizes a sophisticated encoding pipeline.
- Feature Extraction: Images are passed through a backbone (e.g., ResNet-12) to extract high-dimensional features.
- Dimension Reduction: To manage the computational complexity of subsequent steps, Principal Component Analysis (PCA) is used to reduce feature dimensions (e.g., to 80).
- Vector-Kronecker Product: The relationship between the query and support images is captured using a vector-Kronecker product (outer product). The triplet is represented as $(f_q \otimes f_i) - (f_q \otimes f_j)$. This captures correlations across all feature dimensions, providing a richer similarity metric than standard Euclidean or cosine distances.
- RankMLP: The encoded triplet is fed into a multi-layer perceptron (RankMLP) that outputs a binary ranking score.
Experimental Results
RankDNN was evaluated on several standard benchmarks, including miniImageNet, tieredImageNet, CUB, and CIFAR-FS.
- State-of-the-Art Performance: On miniImageNet, RankDNN achieved an accuracy of 68.72% for 1-shot and 85.66% for 5-shot tasks, outperforming established methods like FEAT and IE.
- Cross-Domain Generalization: In the challenging miniImageNet $\to$ CUB task, RankDNN showed a significant improvement of 7.08% over the baseline in the 1-shot setting, reaching 60.64%. This highlights the model's ability to learn transferable ranking rules that apply across disparate datasets.
- Model Fusion: RankDNN is complementary to existing metric-learning methods. When fused with models like ProtoNet or DeepEMD, it consistently improved their baseline performance across all tested scenarios.
Conclusion
RankDNN rethinks few-shot learning by framing it as a relevance ranking problem. By leveraging triplet-based training and Kronecker product encoding, the model avoids the pitfalls of standard classification in low-data regimes. Its superior performance on both within-domain and cross-domain tasks suggests that learning "how to rank" is a powerful paradigm for building generalizable few-shot learners. Future work may focus on optimizing the high-order correlation capture to remove the need for initial dimensionality reduction.
Summary updated