Hyperspectral Anomaly Detection with Self-Supervised Anomaly Prior

Yidan Liu, Weiying Xie, Kai Jiang, Jiaqing Zhang, Yunsong Li, Leyuan Fang

Abstract

The majority of existing hyperspectral anomaly detection (HAD) methods use the low-rank representation (LRR) model to separate the background and anomaly components, where the anomaly component is optimized by handcrafted sparse priors (e.g., $\ell_{2,1}$-norm). However, this may not be ideal since they overlook the spatial structure present in anomalies and make the detection result largely dependent on manually set sparsity. To tackle these problems, we redefine the optimization criterion for the anomaly component in the LRR model with a self-supervised network called self-supervised anomaly prior (SAP). This prior is obtained by the pretext task of self-supervised learning, which is customized to learn the characteristics of hyperspectral anomalies. Specifically, this pretext task is a classification task to distinguish the original hyperspectral image (HSI) and the pseudo-anomaly HSI, where the pseudo-anomaly is generated from the original HSI and designed as a prism with arbitrary polygon bases and arbitrary spectral bands. In addition, a dual-purified strategy is proposed to provide a more refined background representation with an enriched background dictionary, facilitating the separation of anomalies from complex backgrounds. Extensive experiments on various hyperspectral datasets demonstrate that the proposed SAP offers a more accurate and interpretable solution than other advanced HAD methods.

1. Introduction

Hyperspectral Anomaly Detection (HAD) is a critical task in remote sensing, enabling the identification of unusual targets in complex environments. Most traditional HAD methods rely on the Low-Rank Representation (LRR) model, which assumes that a hyperspectral image (HSI) can be decomposed into a low-rank background and a sparse anomaly component. However, these methods typically use handcrafted priors, such as the $\ell_{2,1}$-norm, to enforce sparsity. This approach has significant limitations: it ignores the inherent spatial structure of real-world anomalies (like vehicles or aircraft) and requires sensitive manual tuning of sparsity parameters.

2. Methodology

The proposed framework, Self-Supervised Anomaly Prior (SAP), bridges the gap between classical mathematical modeling and modern deep learning. Instead of relying on rigid mathematical norms, SAP utilizes a deep neural network to learn a data-driven prior for anomalies.

2.1 Self-Supervised Pretext Task

To train the network without human-labeled data, the authors employ a self-supervised pretext task. They generate "pseudo-anomalies" by injecting 3D prisms with arbitrary polygonal bases and random spectral signatures into clean HSI data. A ResNet-34 backbone is then trained to distinguish between the original image and the one containing the pseudo-anomaly. This process forces the network to learn the spatial and spectral characteristics that define an anomaly.

Flowchart of the pretext task for self-supervised learning, showing the generation of pseudo-anomalies.

2.2 Integration and Optimization

The learned prior is integrated into the LRR framework using a Plug-and-Play (PnP) strategy. The trained ResNet-34 is embedded within the Alternating Direction Method of Multipliers (ADMM) optimization loop, acting as the prior that guides the separation of the anomaly component. This allows the model to maintain the robustness of the LRR framework while benefiting from the representational power of deep learning.

Overall schematic of the SAP framework, including dictionary construction, SSL training, and ADMM optimization.

2.3 Dual-Purified Background Dictionary

To ensure the background is modeled accurately, a dual-purified strategy is used to refine the background dictionary. By removing outliers and small clusters that likely contain actual anomalies, the method prevents "leakage" where the background model accidentally learns the targets it is intended to detect.

3. Experimental Results

The SAP method was evaluated against nine state-of-the-art methods on four benchmark datasets: San Diego (SD), Los Angeles (LA), Texas Coast (TC), and Hydice.

Key insights from the experiments include:

  • Superior Detection Accuracy: SAP achieved the highest Area Under the Curve (AUC) for overall detection on the SD, LA, and Hydice datasets.
  • Structural Fidelity: Qualitative results demonstrate that SAP accurately captures the complete morphology of targets, such as the wings and fuselages of airplanes, whereas traditional methods often produce fragmented results.
  • Noise Robustness: In the Texas Coast dataset, SAP successfully distinguished between actual anomalies and sensor "strip noise," a common challenge for sparse-norm methods.

4. Conclusion

The SAP framework offers a more accurate and interpretable solution for HAD by replacing handcrafted sparsity constraints with a self-supervised anomaly prior. By modeling anomalies as structured prisms, the method achieves state-of-the-art performance and reduces dependence on manual parameter tuning. Future work will focus on refining background suppression to further reduce false alarm rates in highly heterogeneous scenes.

Summary updated

Abstract

The majority of existing hyperspectral anomaly detection (HAD) methods use the low-rank representation (LRR) model to separate the background and anomaly components, where the anomaly component is optimized by handcrafted sparse priors (e.g., $\ell_{2,1}$-norm). However, this may not be ideal since they overlook the spatial structure present in anomalies and make the detection result largely dependent on manually set sparsity. To tackle these problems, we redefine the optimization criterion for the anomaly component in the LRR model with a self-supervised network called self-supervised anomaly prior (SAP). This prior is obtained by the pretext task of self-supervised learning, which is customized to learn the characteristics of hyperspectral anomalies. Specifically, this pretext task is a classification task to distinguish the original hyperspectral image (HSI) and the pseudo-anomaly HSI, where the pseudo-anomaly is generated from the original HSI and designed as a prism with arbitrary polygon bases and arbitrary spectral bands. In addition, a dual-purified strategy is proposed to provide a more refined background representation with an enriched background dictionary, facilitating the separation of anomalies from complex backgrounds. Extensive experiments on various hyperspectral datasets demonstrate that the proposed SAP offers a more accurate and interpretable solution than other advanced HAD methods.
BibTeX
@article{liu2025hyperspectral,
  author = {Yidan Liu and Weiying Xie and Kai Jiang and Jiaqing Zhang and Yunsong Li and Leyuan Fang},
  title = {{Hyperspectral Anomaly Detection with Self-Supervised Anomaly Prior}},
  journal = {Neural Networks},
  year = {2025},
  doi = {10.1016/j.neunet.2025.107294},
}
Figure