Equal is Not Always Fair: A New Perspective on Hyperspectral Representation Non-Uniformity

Wuzhou Quan, Mingqiang Wei, Jinhui TangarXiv 2025 · DOI · arXiv

Introduction: The HSI Representation Trilemma

Hyperspectral images (HSIs) provide a rich data source by capturing hundreds of narrow spectral bands. However, this wealth of information comes with inherent non-uniformity. Traditional processing paradigms, such as standard CNNs and Transformers, often treat all dimensions of an HSI cube equally. The authors of "FairHyp" argue that this "one-size-fits-all" approach is fundamentally flawed, leading to biased and suboptimal representations.

The HSI representation trilemma: balancing spatial adaptability, feature efficiency, and spectral long-range dependencies.

The research identifies three core conflicts, termed the "trilemma":

  1. Spatial Variability: Pixel correlations shift based on sensor distance and resolution.
  2. Feature Sparsity: HSIs are naturally low-rank; treating all channels as equally important introduces noise and computational waste.
  3. Spectral Irregularity: Correlation is not always local; distant spectral bands can be more related than adjacent ones due to material properties.

The FairHyp Framework

To address these challenges, the authors propose FairHyp, a modular framework that replaces uniform processing with specialized, "fair" modules.

Overview of the FairHyp framework, featuring the RK4-SVA adapter and specialized paths for restoration and classification tasks.

Specialized Modules

  • RK4-SVA (Spatial Variability Adapter): This module uses a 4th-order Runge-Kutta numerical method to treat spatial degradation as a dynamic process, progressively harmonizing spatial correlations across different imaging resolutions.
  • $S^2$FairConv (Spatial-Spectral Fair Convolution): Designed to handle feature sparsity, this module uses multi-receptive fields and sparse-aware refinement to selectively process information-rich regions while ignoring redundant noise.
  • SCSS (Spectral-Context State Space): Leveraging Mamba (State Space Models), this module captures long-range spectral dependencies efficiently, avoiding the quadratic complexity of standard Transformers.

Evaluation and Results

The FairHyp framework was rigorously tested across four major HSI tasks: classification, denoising, super-resolution, and inpainting.

Performance Highlights

  • Classification: On the Indian Pines dataset, FairHyp achieved an Overall Accuracy (OA) of 98.08%, outperforming specialized Vision Transformer (ViT) models.
  • Restoration: In denoising tasks (ICVL dataset), it reached a PSNR of 42.91 dB, significantly surpassing existing state-of-the-art methods.
  • Efficiency: A Transformer-based variant, FairHyp, maintained high accuracy with only 0.33M parameters*, demonstrating exceptional parameter efficiency.

Conclusion

FairHyp shifts the paradigm of HSI processing from uniformity to fairness. By explicitly modeling the non-uniform nature of hyperspectral data through physics-inspired and state-space modules, it achieves superior generalization and performance across diverse imaging modalities. Future work will look into automating the disentanglement of these non-uniformities and extending the framework to other high-dimensional data like 3D medical imaging.

Summary updated

BibTeX
@misc{quan2025equal,
  author = {Wuzhou Quan and Mingqiang Wei and Jinhui Tang},
  title = {{Equal is Not Always Fair: A New Perspective on Hyperspectral Representation Non-Uniformity}},
  year = {2025},
  eprint = {2505.11267},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2505.11267},
}
Figure