Few-Shot Open-Set Recognition of Hyperspectral Images with Outlier Calibration Network

Debabrata Pal, Valay Bundele, Renuka Sharma, Biplab Banerjee, Yogananda V. Jeppu

Introduction

Hyperspectral Imaging (HSI) provides detailed spectral "fingerprints" for every pixel, enabling precise land-cover classification. However, real-world application is hindered by two primary challenges: the high cost of labeling data (the Few-Shot problem) and the inevitable presence of previously unseen classes in dynamic environments (the Open-Set problem). This paper introduces a novel framework for Few-Shot Open-Set Recognition (FSOSR) that eliminates the need for manual thresholding while significantly improving the identification of both known and unknown classes.

The Challenge: Data Scarcity and the "Unknown"

Standard HSI classification models typically operate under a "closed-set" assumption, where all test samples are assumed to belong to classes seen during training. In reality, a model trained on "Grass" and "Water" will eventually encounter "Buildings" or "Shadows." Closed-set models often misclassify these unknown categories into known boxes with high confidence.

Comparison of known and open-set classes in a hyperspectral scene.

Existing solutions often rely on manual rejection thresholds. However, in few-shot scenarios—where only a handful of labeled samples are available—determining an accurate threshold is nearly impossible due to the lack of statistical representation for class boundaries.

Proposed Methodology

The authors propose a unified architecture designed to handle these constraints through three key innovations:

  1. R3CBAM (Residual 3D Convolutional Block Attention Network): By extending the Convolutional Block Attention Module to 3D, the model captures both rich spectral information and spatial context, effectively denoising the input.
  2. VAE-based Feature Augmentation: A Variational Autoencoder (VAE) is used during meta-learning to generate "pseudo-known" features. This helps the model learn the density and variance of known classes, preventing it from becoming too restrictive around limited training samples.
  3. Outlier Calibration Network (OCN): Instead of manual thresholding, the OCN—a small Multi-Layer Perceptron (MLP)—learns to distinguish between knowns and outliers based on Euclidean distances from class prototypes.
Evolution of learning paradigms leading to the proposed Few-Shot Augmented Open-set Recognition.

Key Results and Insights

The framework was evaluated on four major HSI benchmarks (Indian Pines, Pavia University, Salinas, and Houston-2013) in 1-shot and 5-shot scenarios. The results demonstrate a significant leap over existing methods:

  • Closed-set Overall Accuracy (ClosedOA): Improved by an average of 7%.
  • Open-set Overall Accuracy (OpenOA): Improved by an average of 10%.
  • AUROC: A 20% improvement in the model's ability to distinguish knowns from unknowns.

For instance, in the 5-shot Salinas test, the model achieved an AUROC of 88.52%, vastly outperforming baseline methods like OpenMax (~55%) or PEELER (~60%). These results underscore that treating outlier detection as a learnable task (via OCN) is far superior to static thresholding.

Conclusion

By combining 3D attention, generative augmentation, and learnable outlier calibration, this research provides a robust path forward for HSI classification in data-constrained, unpredictable environments. While the computational weight of 3D convolutions remains a consideration for edge deployment, the shift toward automated, threshold-free recognition marks a significant advancement in the field. Future work will focus on not just rejecting outliers, but actively clustering and identifying them over time.

Summary updated

Abstract

We tackle the few-shot open-set recognition (FSOSR) problem in the context of remote sensing hyperspectral image (HSI) classification. Prior research on OSR mainly considers an empirical threshold on the class prediction scores to reject the outlier samples. Further, recent endeavors in few-shot HSI classification fail to recognize outliers due to the ‘closed-set’ nature of the problem and the fact that the entire class distributions are unknown during training. To this end, we propose to optimize a novel outlier calibration network (OCN) together with a feature extraction module during the meta-training phase. The feature extractor is equipped with a novel residual 3D convolutional block attention network (R3CBAM) for enhanced spectral-spatial feature learning from HSI. Our method rejects the outliers based on OCN prediction scores barring the need for manual thresholding. Finally, we propose to augment the query set with synthesized support set features during the similarity learning stage in order to combat the data scarcity issue of few-shot learning. The superiority of the proposed model is showcased on four benchmark HSI datasets. <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup>
BibTeX
@inproceedings{pal2022few,
  author = {Debabrata Pal and Valay Bundele and Renuka Sharma and Biplab Banerjee and Yogananda V. Jeppu},
  title = {{Few-Shot Open-Set Recognition of Hyperspectral Images with Outlier Calibration Network}},
  booktitle = {IEEE/CVF Winter Conference on Applications of Computer Vision},
  year = {2022},
  doi = {10.1109/wacv51458.2022.00215},
}
Figure