Few-Shot Open-Set Hyperspectral Pixel-Wise Classification with Spatial Aware Label Propagation

Matheus Pereira, Jefersson Santos, Laurent NajmanarXiv 2026 · arXiv

TL;DR

FOSSeL is a novel framework that tackles the "open-set" problem in hyperspectral imaging—identifying both known and unknown land-cover classes with only a handful of labeled pixels. By combining self-supervised feature learning with spatially-aware graph propagation over superpixels, it achieves state-of-the-art performance without requiring massive, expert-labeled datasets.

What Problem Does This Paper Address?

Hyperspectral images (HSI) are incredibly rich, capturing hundreds of spectral bands beyond human vision. However, classifying them is a nightmare for two reasons:

  1. Label Scarcity: Annotating these images requires domain expertise and is time-consuming, leading to a "few-shot" problem where we only have a few labeled pixels per class.
  2. The "Open-World" Reality: Most models assume a "closed-set" scenario—that every pixel at test time belongs to a class seen during training. In the real world, sensors encounter "unknown" classes (e.g., a new type of crop or an unmapped building) that models often misclassify as known ones, tanking their reliability.

Existing methods often struggle to balance these two constraints, especially given the high dimensionality and spatial noise inherent in HSI data.

Methods

The authors propose FOSSeL (Few-shot Open-set hyperspectral recognition from Self-Supervised Learning), a two-stage pipeline that decouples feature learning from classification.

Fig. 1. Overview of the proposed FOSSeL framework. Given a feature space obtained through self-supervised training (see Fig. 2) and a few labeled seed pixels, the open-set prediction module, which can represent several possible algorithms, classifies known pixels and identifies unknown classes, generating the final output.
  • Stage 1: Unsupervised Feature Learning. Instead of using labels, the model uses VICReg (Variance-Invariance-Covariance Regularization) to learn from the raw data. It forces the model to create a discriminative feature space where similar spectral signatures naturally cluster together. This is particularly effective for HSI because it handles highly correlated spectral bands without "representation collapse."
  • Stage 2: Spatially-Aware Open-Set Prediction.
    • Superpixels: The image is first grouped into "superpixels" (clusters of similar, adjacent pixels) using the SLIC algorithm. This drastically reduces the computational load and provides a form of spatial regularization.
  • Graph Propagation: The framework builds a graph where superpixels are nodes. It then "propagates" labels from a few "seed" pixels to the rest of the graph. The authors tested three variants: CSProp (pathfinding-based), SimProp (diffusion-based), and KNNProp (feature-proximity-based).
  • Opening the Set: To identify unknowns, FOSSeL uses simple thresholding. For instance, in CSProp, if a superpixel is too far from any known "seed" in the feature space, it's flagged as "Unknown."

Datasets

The framework was evaluated on three classic HSI benchmarks, adapted for open-set scenarios by hiding specific classes during the "training" phase:

  • Indian Pines (IP): Agricultural land in Indiana; 3 classes (alfalfa, oats, etc.) were treated as unknown.
  • Kennedy Space Center (KSC): Wetland and scrub vegetation; 3 swamp-related classes were hidden.
  • Salinas (SA): California valley imagery; 4 lettuce-related classes were treated as unknown.

Key Insights and Ideas

  • Self-Supervision is a Strong Foundation: One of the biggest "aha" moments is that self-supervised models can organize HSI features into distinct, class-separated clusters without seeing a single label. This "unsupervised clustering" provides a robust map for the later label propagation step.
  • Superpixels are more than a Speedup: While superpixels reduce the graph size by over 98%, they also act as a spatial filter. By forcing labels to propagate through coherent regions, the model becomes much more robust to the "salt-and-pepper" noise common in pixel-wise HSI classification.
  • Decoupling Prevents Overfitting: Because the feature space is learned independently of the labels, the model doesn't overfit to the tiny "few-shot" support set. This makes it more generalizable than supervised methods that try to learn features and classification simultaneously from just 5 or 10 labels.

Key Results

FOSSeL consistently outperformed or matched state-of-the-art supervised methods like OSC-SCL and CACL, particularly in extreme few-shot settings (5-10 labels per class).

  • Accuracy: Using the H-score (a metric that balances accuracy on known classes with the ability to detect unknowns), FOSSeL variants often took the top two spots across all datasets.
  • Efficiency: The superpixel approach makes inference incredibly fast. On the high-resolution KSC dataset, FOSSeL's CSProp method finished in 1.6 seconds, compared to 47 seconds for the supervised CACL baseline.
  • Robustness: The model showed a "rapid saturation" effect—it reached near-peak performance with just 10 labeled samples, suggesting it's highly efficient at extracting information from minimal supervision.

Limitations and Future Work

The authors note that FOSSeL is sensitive to superpixel quality. If a superpixel is too large and "under-segments" the image, it might accidentally merge a known class with an unknown one, leading to errors.

Additionally, the current open-set thresholding is relatively simple (static means or confidence thresholds). Future work could involve more adaptive or learned criteria for identifying "unknowns." The authors also plan to explore domain adaptation, investigating how a model trained on one geographical location might perform on another.

Summary updated

Abstract

Pixelwise classification of hyperspectral images (HSI) is a fundamental task in remote sensing, but its deployment in real-world applications is often limited by the scarcity of labeled data and the presence of unknown classes at inference time. In this work, we propose Few-shot Open-set hyperspectral recognition from Self-Supervised Learning training (FOSSeL), a unified framework for few-shot, open-set hyperspectral pixelwise classification that does not require supervised training. The proposed approach first leverages self-supervised learning to construct a discriminative feature space from unlabeled data. Then, given a small set of labeled seed pixels, class predictions are obtained through spatially-aware graph-based label propagation over superpixel representations, enabling the joint exploitation of spectral similarity and spatial consistency. To address the open-set scenario, we introduce simple yet effective strategies to identify pixels that do not belong to the known classes. We evaluate FOSSeL on three benchmark hyperspectral datasets under multiple few-shot settings. The results show that the proposed framework achieves competitive or superior performance compared to fully supervised methods while maintaining robustness in the presence of unknown classes. These findings highlight the potential of combining self-supervised feature learning with graph-based inference for practical HSI analysis in low-annotation and open-world scenarios.
BibTeX
@misc{pereira2026few,
  author = {Matheus Pereira and Jefersson Santos and Laurent Najman},
  title = {{Few-Shot Open-Set Hyperspectral Pixel-Wise Classification with Spatial Aware Label Propagation}},
  year = {2026},
  eprint = {2611.17974},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.17974},
}
Figure