Explaining Away False Alarms: Neighborhood Material Consistency for Hyperspectral Anomaly Detection

A hyperspectral camera records hundreds of narrow color bands per pixel. That is enough to tell painted metal from sand by chemistry rather than by appearance. Anomaly detectors on this data look for the handful of pixels, often under 0.1% of a scene, that do not match their surroundings. On standard scenes these detectors already score near-perfect area under the ROC curve (AUC). In practice they still fire on benign oddities: a shadowed patch of beach, a stripe of sensor noise, an unusual rooftop. With anomalies this rare, even a tiny false-positive rate swamps the true detections, and AUC averages that failure away. Our proposal is to score each region of the scene by whether its neighbors can explain it in a shared vocabulary of named materials and conditions, not by how numerically far it sits from them. "Sand in shadow" next to "sand" is explained and should stay silent. "Painted metal, sub-pixel" in the middle of a beach is not explained and should fire.

To learn this without labeled anomalies, we generate training data from language. A language model writes short descriptions of scene-plausible conditions (shadow, haze, noise stripes, moisture) and scene-foreign targets (a vehicle-sized metal object on sand). Each description is turned into a physically mixed spectral-spatial patch using measured material spectra. The detector is trained so that the spectrum of a region and the context of its graph neighbors agree on which description applies. The descriptions do double duty. They are the source of realistic pseudo-anomalies. They also act as a semantic label space that makes the detector invariant to benign changes. The bet is that this cuts false alarms at the operating points that matter, where current detectors that only measure numerical distance cannot.

Core Claim

Under extreme imbalance, false alarms come mostly from rare-but-explainable background, not from missed targets. A detector that asks whether a region's material is consistent with its neighborhood's material vocabulary will beat purely numerical spectral-spatial detectors on detection probability at very low false-alarm rates, while matching them on AUC. The language component earns its place only if it beats the same pipeline with named spectra sampled directly from a library, with no descriptions. We treat that comparison as the decisive test, not a side ablation.

How This Sits Relative to Prior Work

Hyperspectral anomaly detection supplies both the problem and the strongest competitors. SSIF multiplies an Isolation Forest score on superpixels by a local spatial saliency score, then smooths with an edge-preserving filter. It reaches 0.9978 AUC on Beach, yet loses 14.31% AUC under shadow, which is exactly the benign-variation failure we target. SAP learns an anomaly prior by pasting random polygonal prisms with random spectral curves into clean cubes. Its authors concede these do not resemble real targets. We keep SAP's idea of self-supervised pseudo-anomalies but replace random geometry and spectra with described, physically mixed ones. We also add hard negatives, meaning benign conditions, which SAP does not generate. STAD motivates abandoning reconstruction error, since autoencoders reconstruct small anomalies too well. It reaches 0.9899 average AUC on HAD100 using feature-space saliency. STAD is our main numerical competitor.

The scoring mechanism comes from anomaly detection on text-attributed graphs, which are graphs whose nodes carry text. CMUCL trains a text encoder and a graph network jointly. It scores nodes by the disagreement between a node's own representation and its neighborhood's, reporting an 11.13% gain in average precision over the next best method. We transfer this cross-view inconsistency score to a graph of hyperspectral superpixels. We change what the two views must agree on: a distribution over material descriptions, rather than an unconstrained embedding. TAG-AD motivates using language at all. It finds language models far better than graph models at contextual anomalies (0.966 vs 0.688 ROC-AUC on Cora). It also constrains us: language models are "graph-blind" on structural anomalies. So we never ask a language model to read the scene. Language is used only offline, to write descriptions, and all spatial reasoning stays in a graph network.

The node encoder borrows from spectral-spatial fusion work. FactoFormer motivates factorized spectral and spatial attention over monolithic 3D convolutions. AHSGC builds superpixel graphs with an adaptive filter that balances smoothing against high-frequency detail. That balance matters because plain smoothing would erase the sub-pixel targets STAD worries about.

Remote-sensing vision-language work such as VRSBench is RGB-only, so it cannot be used off the shelf here.

Research Program

We build three pieces.

The description-driven generator. Given a one-line scene caption written once per training scene (e.g. "coastal beach, sand and water"), a language model produces two kinds of descriptions. Benign descriptions name a background material plus a condition, such as shadow depth, haze, moisture, or a band-range noise stripe. Foreign descriptions name a material absent from the caption plus a size, often sub-pixel, and a shape. A deterministic renderer turns each description into pixels. Materials map to laboratory spectra, and illumination conditions become physically motivated spectral transforms. Size becomes a linear mixing fraction with the local background, and shape becomes a spatial mask. The renderer is the auditable part: every synthetic patch carries its description as a label.

The detector. Each scene is segmented into superpixels that form graph nodes, with edges for spatial adjacency. A factorized spectral-spatial encoder embeds each superpixel. A frozen sentence encoder embeds the vocabulary of material-and-condition descriptions. Two heads predict a distribution over descriptions. The first uses the node alone, giving $p(m \mid x_v)$, where $m$ is a description and $x_v$ the features of node $v$. The second uses a graph network over the neighbors of $v$, giving $q(m \mid N(v))$, where $N(v)$ is the neighbor set. Both heads are trained on synthetic labels. A contrastive term in the style of CMUCL prevents the node head from ignoring the neighborhood, a failure that GraphFormers documents as the lazy-encoder shortcut. The anomaly score is

$$s(v) = -\log \sum_m p(m \mid x_v)\, q(m \mid N(v)) \;+\; \lambda\, H\big(p(\cdot \mid x_v)\big),$$

The first term is large when node and neighborhood disagree on the material. The second term, the entropy $H$ of the node's own description distribution weighted by $\lambda$, catches spectra the vocabulary cannot name at all. A shadowed sand node gets a confident "sand, shadowed" prediction that its sandy neighbors also support, so it scores low. A foreign metal node either gets a confident "painted metal" prediction the neighborhood rejects, or a diffuse prediction. Either way it scores high.

The analysis. Every false alarm at the fixed operating point is categorized: shadow, noise stripe, boundary, rare background material, or other. The aim is to show which false alarms the method removes, not just that the average went down.

A known risk is that real targets can be semantically plausible, like aircraft at an airport in San Diego. For this reason the vocabulary consistency is defined at the material level, not the object level. "Aircraft paint" remains foreign to a "concrete, asphalt, grass" neighborhood. We will check this case explicitly.

Baselines

The naive reference is the Reed-Xiaoli detector (RX), which scores each pixel by its Mahalanobis distance from global background statistics. It has no learning and no spatial context. SSIF reports RX at 0.9807 AUC on Beach, and we rerun it on every scene.

The strongest classical fused detector is SSIF, with a reported 0.9978 AUC on Beach. We rerun it with its published pipeline on all scenes so that low-false-alarm metrics, which it does not report, are available.

The strongest learned competitor is STAD. It reports 0.9899 average AUC and 0.9836 background-suppression AUC on HAD100, failing on 2 of 94 test images. We train it and our model on the same 260 background-only HAD100 training images, using the same first 50 bands, the same GPU-hour budget, and the same number of hyperparameter trials. Hyperparameters for both are selected only on held-out training scenes containing synthetic anomalies, never on test masks. Beating STAD means matching its AUC within noise and raising detection probability at a false-alarm rate of 1e-3 by a margin that holds across three seeds and a paired bootstrap over test images.

SAP is the closest pseudo-anomaly method. It reports the best overall AUC on San Diego, Los Angeles and Hydice, but no reported numeric result was found on Lacuna for SAP. We therefore run SAP's prism generator inside our own detector. This is also our first ablation: it isolates synthesis quality from architecture.

Two further ablations isolate the key ingredients. The library-only generator samples named spectra and conditions at random, with no language model. The text-free variant replaces the description heads with unconstrained embeddings and scores raw node-versus-neighborhood embedding distance, as CMUCL would on this graph.

Why not just…?

Why not just sample material spectra from a library and skip the language model? This is the most serious threat. A spectral library already names materials, and random draws of material, condition, and size might cover the same space. A language model adds two things: scene-conditioned pairing of benign conditions with the materials actually present, and plausible foreign-object combinations. These might not matter. The library-only ablation, run with identical rendering and training, settles it. If it matches the full system at false-alarm rate 1e-3 on HAD100 and the classic scenes, we drop the language model and report the method as vocabulary-consistency scoring with library synthesis. The core claim about explaining away false alarms survives either way.

Why not just compare each node to its neighbors numerically, as CMUCL does, without any vocabulary? A raw embedding distance cannot tell "different because shadowed" from "different because foreign." Both move the spectrum. The vocabulary is what ties shadowed sand to sand. The text-free ablation tests this directly. Our prediction is that it helps on AUC but not on the shadow and stripe subsets. We read the outcome from the false-alarm categorization, not the average. If the text-free variant removes shadow false alarms equally well, the semantic label space is unnecessary.

Why not just post-filter an existing detector, or tune its threshold? SSIF already applies edge-preserving filtering after fusion and still loses 14.31% AUC under shadow. That suggests smoothing cannot distinguish a benign spectral shift from a target. We nonetheless apply the same filter to STAD's output as an extra baseline, so the gain cannot be attributed to post-processing.

Why not just feed a remote-sensing vision-language model? VRSBench models are RGB-only, and GeoChat reaches only 40.8% VQA accuracy before in-domain fine-tuning. Rendering the cube to RGB discards the very bands that separate camouflage from background, a separation SDCM exploits for 93.6% [email protected] on HOD-1. We include a frozen-RGB-embedding-plus-RX baseline only as a sanity floor.

Proposed Experiments

The primary benchmark is HAD100, because it alone in the file provides a background-only training split. That split makes self-supervised training and fair tuning possible. The secondary benchmarks are the classic AVIRIS and ROSIS scenes: Beach, San Diego, Gulfport, Pavia City, Los Angeles, Texas Coast and Hydice. On these we use a train-on-HAD100, test-transfer protocol, with no per-scene tuning.

We report AUC and background-suppression AUC for comparability. The headline metrics are detection probability at false-alarm rates of 1e-3 and 1e-4, partial AUC below 1e-3, and average precision. All of these reflect performance when targets are under 0.1% of pixels.

Three checks decide the proposal. First, on HAD100 the full method must beat STAD at false-alarm rate 1e-3 while staying within noise on AUC. Second, on shadowed regions and the stripe-noise scene Texas Coast, false alarms must drop relative to STAD, SSIF and the text-free ablation, with the reduction concentrated in the shadow and stripe categories. Third, the full method must beat library-only synthesis. Failing that third check reshapes the contribution rather than ending it.

As a stress test, we treat HOD-1's camouflaged objects as anomalies and score them without box supervision. This asks whether described foreign materials transfer to targets hidden against nearly identical spatial backgrounds.