Hyperspectral anomaly detection with self-supervised anomaly prior

Yidan Liu, Kai Jiang, Weiying Xie, Jiaqing Zhang, Yunsong Li, Leyuan Fang

Hyperspectral Anomaly Detection (HAD) is a critical task in remote sensing, used for everything from environmental monitoring to defense. However, traditional methods often struggle to distinguish between actual targets and complex backgrounds. A new paper, "Hyperspectral Anomaly Detection with Self-Supervised Anomaly Prior," proposes a clever way to bridge the gap between classical mathematical modeling and modern deep learning.

TL;DR

The authors replace handcrafted mathematical "sparsity" constraints in hyperspectral anomaly detection with a Self-Supervised Anomaly Prior (SAP). By training a deep network to recognize "pseudo-anomalies" generated from unlabeled data, they achieve state-of-the-art detection accuracy that respects the actual spatial shapes of targets.

What Problem Does This Paper Address?

Hyperspectral images (HSI) capture hundreds of narrow spectral bands for every pixel, providing a "spectral signature" for materials. Most HAD methods use a Low-Rank Representation (LRR) model, which assumes an image can be split into a low-rank background and a sparse anomaly component.

The "gap" lies in how we define "anomaly." Traditionally, researchers use handcrafted priors like the $\ell_{2,1}$-norm to force the anomaly component to be sparse. This has two major flaws:

  1. Ignores Structure: Real-world anomalies (like a plane on a runway) are structured objects, not just isolated, random pixels. Handcrafted norms treat them as the latter.
  2. Manual Tuning: These models rely heavily on a "sparsity parameter" ($\beta$) that must be manually tuned for every different scene, making them brittle in practice.

Methods

The researchers introduce a framework called SAP (Self-supervised Anomaly Prior). The core idea is to use a Deep Neural Network (DNN) to learn what an anomaly "looks like" without needing human-labeled training data.

1. The Pretext Task (Learning the Prior)

To train the network without labels, the authors use Self-Supervised Learning (SSL). They take "clean" hyperspectral data and inject pseudo-anomalies. As shown in Figure 2, these are designed as 3D prisms with arbitrary polygonal bases and random spectral bands. The flowchart of the pretext task for self-supervised learning. A ResNet-34 backbone is then trained on a simple classification task: "Is this the original image, or does it contain a pseudo-anomaly?" This forces the network to learn the spatial and spectral characteristics of unusual objects.

The flowchart of the pretext task for self-supervised learning.

2. Integration via Plug-and-Play

Instead of a standard end-to-end black box, the authors use a Plug-and-Play (PnP) strategy. They keep the robust LRR mathematical framework but "plug" their trained ResNet into the optimization loop (specifically the ADMM algorithm). The network acts as the "prior" that guides the separation of the anomaly component.

3. Dual-Purified Background Dictionary

To ensure the background is modeled accurately, they propose a dual-purified strategy. This cleans the "dictionary" of pixels used to represent the background by removing outliers and small clusters that likely contain actual anomalies, preventing "leakage" where the background model accidentally learns the targets it's supposed to detect.

Schematic of the proposed SAP method, showing the dictionary construction, SSL training, and LRR optimization.

Datasets

The model was evaluated on four widely recognized HAD benchmarks:

  • San Diego (SD): High-resolution data featuring airplanes.
  • Los Angeles (LA): Urban scene with airplanes and complex taxiways.
  • Texas Coast (TC): Coastal scene with houses and significant "strip noise."
  • Hydice: A classic dataset featuring small vehicles in a rural setting.

For the self-supervised training phase, they used a large, separate dataset of 1,385 hyperspectral images to ensure the network generalized well before being applied to the test cases.

Key Insights and Ideas

  • Anomalies are Prisms: By modeling pseudo-anomalies as prisms with polygonal bases, the authors effectively mimic the "unexpectedness" and "irregularity" of real-world targets in both space and spectrum.
  • Beyond Sparsity: The move from "pixel-wise sparsity" to "deep priors" allows the model to recognize that if five pixels in a row look like a wing, the sixth pixel is likely part of the same anomaly, even if its spectral signature is slightly different.
  • Parameter Independence: Unlike traditional LRR, SAP is largely independent of manually set sparsity parameters, making it much more "set-and-forget" for practical applications.

Key Results

The SAP method outperformed nine other state-of-the-art methods (including both classical LRR and newer DNN-based approaches) across almost all metrics:

  • Accuracy: SAP achieved the highest Area Under the Curve (AUC) for overall detection on the SD, LA, and Hydice datasets.
  • Visual Clarity: Qualitative results showed that SAP produces "cleaner" detection maps. While other methods often flagged edges of buildings or sensor noise as anomalies, SAP correctly identified the complete morphology of targets like airplanes.
  • Noise Robustness: In the Texas Coast dataset, which is plagued by sensor "strip noise," SAP successfully ignored the noise while other sparse-norm methods mistakenly flagged the strips as anomalies.

Limitations and Future Work

While SAP is highly effective at identifying targets, the authors noted that on certain datasets (like Texas Coast), the false alarm rate was slightly higher than some competitors. The "background suppression" isn't always perfect when the background is extremely heterogeneous.

The authors suggest that future research should focus on further refining the background representation to reduce these false positives without sacrificing the model's ability to detect structured targets.

Summary updated

Abstract

The majority of existing hyperspectral anomaly detection (HAD) methods use the low-rank representation (LRR) model to separate the background and anomaly components, where the anomaly component is optimized by handcrafted sparse priors (e.g., $\ell_{2,1}$-norm). However, this may not be ideal since they overlook the spatial structure present in anomalies and make the detection result largely dependent on manually set sparsity. To tackle these problems, we redefine the optimization criterion for the anomaly component in the LRR model with a self-supervised network called self-supervised anomaly prior (SAP). This prior is obtained by the pretext task of self-supervised learning, which is customized to learn the characteristics of hyperspectral anomalies. Specifically, this pretext task is a classification task to distinguish the original hyperspectral image (HSI) and the pseudo-anomaly HSI, where the pseudo-anomaly is generated from the original HSI and designed as a prism with arbitrary polygon bases and arbitrary spectral bands. In addition, a dual-purified strategy is proposed to provide a more refined background representation with an enriched background dictionary, facilitating the separation of anomalies from complex backgrounds. Extensive experiments on various hyperspectral datasets demonstrate that the proposed SAP offers a more accurate and interpretable solution than other advanced HAD methods.
BibTeX
@article{liu2025hyperspectral,
  author = {Yidan Liu and Kai Jiang and Weiying Xie and Jiaqing Zhang and Yunsong Li and Leyuan Fang},
  title = {{Hyperspectral anomaly detection with self-supervised anomaly prior}},
  journal = {Neural Networks},
  year = {2025},
  doi = {10.1016/j.neunet.2025.107294},
}

Loading related content...

Figure