Boosting Few-Shot Open-Set Recognition with Multi-Relation Margin Loss

Yongjuan Che, Yuexuan An, Hui Xue

Abstract

Few-Shot Open-Set Recognition (FSOSR) presents a significant challenge: models must classify known categories with minimal examples while effectively rejecting unknown classes. We present the Multi-Relation Margin (MRM) loss, a novel "plug-and-play" module that enhances feature discriminability by learning hypersphere boundaries and modeling complex pair-wise relationships. Unlike methods that rely on synthetic "fake" unknowns, MRM implicitly defines a safety zone for known data. Experimental results on benchmarks like miniImageNet show substantial improvements in AUROC, demonstrating the effectiveness of margin-based boundary refinement in few-shot scenarios.

Introduction

Standard Few-Shot Learning (FSL) typically operates under a closed-set assumption, where all test samples belong to the small set of classes seen during training. However, real-world applications often encounter "unknown" categories. Few-Shot Open-Set Recognition (FSOSR) requires models to not only classify known categories accurately but also to identify and reject out-of-distribution samples.

The primary difficulty in FSOSR lies in defining precise class boundaries with only 1 or 5 examples per class. Existing approaches often produce "blurry" decision boundaries or depend on generating pseudo-unknown samples, which can be computationally expensive or of poor quality. This work proposes to sharpen the boundaries of known classes to implicitly carve out the space where unknowns likely reside.

Methodology: Multi-Relation Margin Loss

The Multi-Relation Margin (MRM) loss is designed to be integrated into existing few-shot architectures such as ProtoNet or FEAT. It employs three core strategies to refine the feature space:

  1. Anchor Generation: The model utilizes class prototypes—the mean feature vector of available samples—as anchors to represent the global distribution of each category.
  2. Margin and Radius Refining: Instead of collapsing samples to a single point, MRM learns a hypersphere for each class with a learnable radius ($R_c$). The loss function constrains known samples within this sphere while pushing samples from other classes beyond a safety margin ($m$).
  3. Pair-Wise Relation Modeling: By calculating weights based on self-relation (similarity to class anchor) and relative relation (comparison with other pairs in a batch), the model prioritizes "hard" samples near the decision boundaries.
Comparison of different loss functions: (a) Triplet loss, (b) Center loss, (c) Multi-similarity loss, and (d) the proposed MRM loss.

By preserving the internal similarity structure within the hypersphere, MRM avoids the over-compression of features that often plagues traditional few-shot methods. This tighter, more distinct boundary naturally facilitates the detection of unknown objects in the "empty space" between classes.

Experiments and Results

The effectiveness of MRM loss was evaluated on three standard computer vision benchmarks: CUB-200, tieredImageNet, and miniImageNet. The experiments followed a "5-way" setup (5 known classes) with 1-shot or 5-shot support sets, introducing 5 additional unknown classes during the testing phase.

The MRM loss demonstrated significant performance gains, particularly in the Area Under the Receiver Operating Characteristic (AUROC) curve, which measures the ability to detect unknowns:

  • AUROC Improvements: On miniImageNet (1-shot), adding MRM to a baseline ProtoNet increased the AUROC from 51.81% to 68.11%—a nearly 16-point jump.
  • State-of-the-Art Performance: When integrated with advanced models like FEAT, the MRM-enhanced versions outperformed specialized FSOSR methods such as SnaTCHer and ProCAM across almost all benchmarks.
  • Classification Accuracy: Importantly, the improved rejection capability did not come at the cost of classification accuracy; in many instances, closed-set accuracy also improved due to a more organized feature space.

Conclusion and Future Work

The Multi-Relation Margin loss provides a robust mechanism for improving FSOSR by focusing on the geometry of the known class distributions. While highly effective, the method's performance is somewhat dependent on the quality of the pre-trained backbone (e.g., ResNet12) and requires careful tuning of hyperparameters like scaling factors and the margin.

Future research may explore dynamic adaptation of the hypersphere radius or the integration of this margin-based approach with generative models to explicitly model the "unknown" space between hyperspheres.

Summary updated

Abstract

Few-shot open-set recognition (FSOSR) has become a great challenge, which requires classifying known classes and rejecting the unknown ones with only limited samples. Existing FSOSR methods mainly construct an ambiguous distribution of known classes from scarce known samples without considering the latent distribution information of unknowns, which degrades the performance of open-set recognition. To address this issue, we propose a novel loss function called multi-relation margin (MRM) loss that can plug in few-shot methods to boost the performance of FSOSR. MRM enlarges the margin between different classes by extracting the multi-relationship of paired samples to dynamically refine the decision boundary for known classes and implicitly delineate the distribution of unknowns. Specifically, MRM separates the classes by enforcing a margin while concentrating samples of the same class on a hypersphere with a learnable radius. In order to better capture the distribution information of each class, MRM extracts the similarity and correlations among paired samples, ameliorating the optimization of the margin and radius. Experiments on public benchmarks reveal that methods with MRM loss can improve the unknown detection of AUROC by a significant margin while correctly classifying the known classes.
BibTeX
@inproceedings{che2023boosting,
  author = {Yongjuan Che and Yuexuan An and Hui Xue},
  title = {{Boosting Few-Shot Open-Set Recognition with Multi-Relation Margin Loss}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2023},
  doi = {10.24963/ijcai.2023/390},
}
Figure