Probabilistic von Mises–Fisher Representation Learning forFew-Shot Remote Sensing Scene Classification

Zhong Ji, Ci Liu, Hongsheng Zhang, Chen Tang, Yanwei PangResearch Square 2026 · DOI

Abstract

Few-Shot Remote Sensing Image Scene Classification (FS-RSISC) remains challenging due to scarce labeled data, severe intra-class variation, and high inter-class similarity inherent in remote sensing imagery. To address these issues, we propose a probabilistic reformulation of FS-RSISC that moves beyond rigid point-based prototypes by modeling each class as a probability distribution over the hyperspherical feature space. Specifically, we recast category representation as a density estimation problem on the unit hypersphere and adopt the von Mises–Fisher (vMF) distribution to capture both semantic uncertainty and feature diversity. We develop a Bayesian-inspired parameter estimation pipeline that initializes vMF parameters using CLIP-derived cross-modal embeddings as semantic priors and progressively refines them via data augmentation and a Mixture-of-Experts based adapter tuning mechanism, enabling robust adaptation under few-shot constraints. Furthermore, we propose two loss functions specifically formulated for the vMF distribution that exploit the geometry of the hypersphere: an Intra-Class Compactness (ICC) loss that leverages the concentration parameter to impose angularly adaptive regularization and enhance feature concentration around semantic centroids, and an Inter-Class Repulsion (ICR) loss that explicitly maximizes angular separation between class centroids to mitigate inter-class overlap. Extensive experiments on three standard benchmarks demonstrate that our method consistently outperforms existing state-of-the-art approaches, establishing a principled and effective framework for FS-RSISC.

The Challenge of Few-Shot Remote Sensing

Few-Shot Remote Sensing Image Scene Classification (FS-RSISC) aims to categorize aerial or satellite images using only a handful of labeled examples. This task is notoriously difficult due to two primary factors: severe intra-class variation and high inter-class similarity. For example, a "residential" neighborhood in one geographic region may look vastly different from one in another, yet it might appear nearly identical to a "commercial" zone from a bird's-eye view.

Example of a 'Residential' scene in remote sensing imagery.

Recently, Vision-Language Models (VLMs) like CLIP have been adapted for this task. However, standard adaptation methods represent each class as a single, rigid point (a prototype) in the feature space. When a class is highly diverse, a single point cannot adequately cover its feature manifold, leading to misalignment between decision boundaries and the actual data distribution.

Example of a complex building scene, illustrating the visual complexity and potential for inter-class confusion.

Methodology: Probabilistic vMF Representation

To overcome the rigidity of point-based prototypes, this research proposes a probabilistic reformulation of the classification task.

vMF Distribution Modeling

Instead of a single point, each class is modeled as a von Mises-Fisher (vMF) distribution—the canonical probability distribution for directional data on a unit hypersphere. This approach captures both the semantic center (mean direction) and the feature diversity (concentration) of each category, naturally accommodating semantic uncertainty.

Bayesian-Inspired Parameter Estimation

Because few-shot learning lacks the data to accurately estimate distribution parameters from scratch, the model initializes vMF parameters using CLIP's cross-modal embeddings (combining text and image features) as a strong semantic prior. These parameters are progressively refined using a Mixture-of-Experts (MoE) adapter tuning mechanism, providing a stable, semantically grounded anchor for the class centroid.

Geometric Loss Functions

The framework introduces two specialized loss functions designed for the hyperspherical feature space:

  • Intra-Class Compactness (ICC) Loss: This loss replaces standard cosine similarity with a vMF-driven metric. It dynamically tightens feature clusters based on how close a class is to its neighbors. By utilizing a heavy-tailed Student-t distribution profile, it ensures that outlier samples still generate useful training gradients.
  • Inter-Class Repulsion (ICR) Loss: This loss explicitly pushes class centroids apart on the hypersphere. It employs an entropy-regularized weighting scheme to apply stronger repulsive forces to classes that are easily confused with one another.

Experimental Results

The framework was evaluated on three standard public remote sensing benchmarks: WHU-RS19, UC Merced (UC), and NWPU-RESISC45 (NWPU).

Performance and Efficiency

The vMF-CLIP framework consistently outperformed existing Parameter-Efficient Fine-Tuning (PEFT) methods (such as CoOp and MaPLe) and remote-sensing-specific adaptations (like RemoteCLIP). Key quantitative gains include:

  • Accuracy Improvements: Top-1 accuracy improved by 2.67%–6.25% on UC, 0.38%–2.48% on NWPU, and 1.75%–3.51% on WHU across various few-shot settings (1 to 16 shots).
  • Model Efficiency: With only ~0.33M trainable parameters, this lightweight adapter outperformed full fine-tuning of the entire CLIP model (~149M parameters) in multiple scenarios.
  • Qualitative Validation: UMAP visualizations confirmed that the combination of ICC and ICR losses resulted in significantly tighter intra-class clusters and wider margins between different classes compared to standard cross-entropy baselines.

Conclusion

By modeling classes as probability distributions rather than fixed points, this research establishes a principled foundation for few-shot remote sensing classification. The integration of vMF distributions with cross-modal priors and geometric losses effectively addresses the inherent variability of aerial imagery. This probabilistic density estimation framework holds potential for expansion into other multimodal applications, such as segmentation and change detection.

Summary updated

Abstract

<title>Abstract</title> Few-Shot Remote Sensing Image Scene Classification (FS-RSISC) remains challenging due to scarce labeled data, severe intra-class variation, and high inter-class similarity inherent in remote sensing imagery. To address these issues, we propose a probabilistic reformulation of FS-RSISC that moves beyond rigid point-based prototypes by modeling each class as a probability distribution over the hyperspherical feature space. Specifically, we recast category representation as a density estimation problem on the unit hypersphere and adopt the von Mises–Fisher (vMF) distribution to capture both semantic uncertainty and feature diversity. We develop a Bayesian-inspired parameter estimation pipeline that initializes vMF parameters using CLIP-derived cross-modal embeddings as semantic priors and progressively refines them via data augmentation and a Mixture-of-Experts based adapter tuning mechanism, enabling robust adaptation under few-shot constraints. Furthermore, we propose two loss functions specifically formulated for the vMF distribution that exploit the geometry of the hypersphere: an Intra-Class Compactness (ICC) loss that leverages the concentration parameter to impose angularly adaptive regularization and enhance feature concentration around semantic centroids, and an Inter-Class Repulsion (ICR) loss that explicitly maximizes angular separation between class centroids to mitigate inter-class overlap. Extensive experiments on three standard benchmarks demonstrate that our method consistently outperforms existing state-of-the-art approaches, establishing a principled and effective framework for FS-RSISC.
BibTeX
@inproceedings{ji2026probabilistic,
  author = {Zhong Ji and Ci Liu and Hongsheng Zhang and Chen Tang and Yanwei Pang},
  title = {{Probabilistic von Mises–Fisher Representation Learning forFew-Shot Remote Sensing Scene Classification}},
  booktitle = {Research Square},
  year = {2026},
  doi = {10.21203/rs.3.rs-8260439/v1},
}
Figure