Two Kinds of "I Don't Know": Separating Mixed Pixels from Unknown Materials in Hyperspectral Classification

A hyperspectral camera records each pixel as a curve of a few hundred reflectance values across narrow wavelength bands. That detail is what lets it tell two crop varieties or two minerals apart. Classifiers on these images still give every pixel one hard label. When they hesitate, they hesitate for two very different reasons that call for different follow-up. A mixed pixel covers several known materials, for example grass at the edge of a roof. Its spectrum supports two classes at once, and the right response is to run spectral unmixing or report a set of labels. An unknown material is a pixel unlike anything in the training set. Its spectrum supports no class, and the right response is to send someone to the field or add a new class. A single confidence score, such as the top softmax probability or the total uncertainty of a Dirichlet distribution over class probabilities, cannot say which of the two is happening.

The proposal is to have the classifier report, for every class, three separate quantities: how much the spectrum supports that class, how much it explicitly contradicts it, and how much of the spectrum is uninformative. A neutrosophic set, an extension of fuzzy sets, stores exactly these three independent degrees of truth, falsity and indeterminacy. Each degree is computed separately over short groups of adjacent wavelength bands instead of over the full few-hundred-dimensional spectrum. A cost-sensitive three-way decision rule then uses these degrees to choose among three kinds of action: accept a label, defer the pixel as mixed, or defer it as unknown. Working band-group by band-group is also how the design handles the curse of dimensionality. Each piece of evidence is estimated in a space of roughly ten to twenty dimensions, which a few hundred labels can actually fill. Disagreement between band groups carries exactly the signal that separates a mixture from a novelty.

Core Claim

Two failure modes look alike to current uncertainty-aware hyperspectral classifiers, and the claim is that they can be told apart. Mixed pixels produce strong support for two classes coming from different spectral regions. Unknown materials produce strong contradiction for all known classes. The two signatures can be separated only if support and contradiction are estimated independently rather than tied together as $\nu = 1 - \mu$. Here $\mu$ is the degree of membership in a class and $\nu$ the degree of non-membership. The softmax enforces this coupling across classes and Zadeh fuzzy sets enforce it within one class.

We predict three things. First, a classifier that keeps the three degrees separate will route mixed and unknown pixels to the correct deferral action more accurately than evidential networks that split uncertainty into "dissonance" and "vacuity". Second, it will do so in a single forward pass. Third, its advantage will grow as labels get scarcer relative to the number of bands, which is the regime where the Hughes phenomenon degrades density-based estimates of ignorance. The Hughes phenomenon is the drop in accuracy when bands are added without adding training samples (Sujuan Wang et al.).

Where This Sits

The closest work, and the one to beat, is Uncertainty-aware Graph-based Hyperspectral Image Classification. It places Dirichlet priors over class probabilities in graph networks, adds a regularizer based on linear-unmixing reconstruction error and a spatial smoothness term, and raises out-of-distribution AUROC on Pavia University from 57.04% (softmax GCN) to 94.55%. This paper motivates the proposal in two ways. It shows that softmax training behaves like linear discriminant analysis and assigns high confidence to unseen materials. It also shows that physical spectral knowledge helps. The proposal departs from it in three respects. That work evaluates uncertainty only post hoc, as AUROC curves, and never makes a decision. It treats a single held-out class as the only kind of novelty. It has no explicit target for mixed pixels at all. Its unmixing regularizer also needs a complete, linear endmember library, and it concedes that this is often unavailable.

Three-way decision supplies the decision side of the method. Decision-theoretic rough sets derive accept, reject and defer thresholds from a loss matrix. 3W-DBFLDM shows that dual thresholds cut total decision cost on tabular credit data from 10,021 to 7,049, but it is strictly binary and solved by kernel quadratic programming. TWDBDL applies entropy-threshold deferral to RGB skin images using four-network ensembles plus Monte Carlo dropout. That is affordable for 660 test images and not for a million-pixel scene. All of these methods defer into a single boundary region. We split that region into two actionable ones, and we avoid the explosion of $3^K$ regions that worries the fuzzy three-way literature by keeping $K+2$ actions per pixel.

The nuanced-uncertainty literature supplies the representation and sets the constraints on it. IF-BLS computes independent membership and non-membership degrees per training sample, beating plain fuzzy weighting on 28 UCI datasets (84.51% vs 82.54% average accuracy). However, it uses them only to reweight training and still predicts with argmax. Pythagorean fuzzy rough sets point out that intuitionistic fuzzy sets, which require $\mu+\nu \le 1$, must truncate cases with strong evidence on both sides, such as $\mu=0.8, \nu=0.4$. That case is exactly the mixed-pixel signature. Neutrosophic three-way decision removes the coupling entirely, but it has only been tested on UCI tables with at most 13 attributes and requires manual discretization. We reuse the representation and give it a learned, spectrally grounded source for each degree. That source is what these tabular methods lack, and it is what makes the representation meaningful at 200 bands.

Research Program

The model has a shared spectral-spatial encoder. We plan to use SpectralMamba because its cost grows linearly in the number of bands and it reaches 89.52% overall accuracy on Houston2013 with 36.55K parameters. The encoder emits one embedding for each of $G$ contiguous band groups, built like the overlapping group-wise embeddings of SpectralFormer so that local absorption slopes stay intact. Each group $g$ has two readouts per class $k$, and they come from deliberately different evidence.

The support $s_{gk}$ is a sigmoid score trained one-vs-rest to recognize class $k$ from that group. The contradiction $c_{gk}$ is computed without a learned classifier. It measures how far the group embedding falls outside the envelope of class $k$'s training embeddings in that group: zero inside a per-class radius, rising smoothly outside it. The radius is fit on a held-out calibration split.

Because the two signals come from different mechanisms, a pixel can have high support and high contradiction for the same class from different groups, or neither. This is the separation a softmax cannot produce. Per-pixel neutrosophic degrees are then aggregated across groups as follows:

$$T_k = \max_g s_{gk},\qquad F_k = \tfrac{1}{G}\sum_g c_{gk},\qquad I = \tfrac{1}{G}\sum_g \mathbf{1}\!\left[\max_k s_{gk} < \tau_s \ \wedge\ \min_k c_{gk} < \tau_c\right].$$

$T_k$ is how strongly some spectral region vouches for class $k$. $F_k$ is how consistently the spectrum lies outside class $k$'s range. $I$ is the fraction of groups that neither support nor exclude any class, for example noisy water-absorption bands or shadowed pixels. The thresholds $\tau_s, \tau_c$ are set on the calibration split.

A pure pixel shows one high $T$ and high $F$ for every other class. A mixed pixel shows two high $T$ values coming from disjoint groups. An unknown material shows high $F$ for every class.

The decision rule picks, for each pixel, the action with the lowest expected cost among K+2 options: accept label $k$, defer-as-mixed, or defer-as-unknown. The cost matrix assigns separate prices to a wrong label, a wasted unmixing call, and a wasted field visit. The decision thresholds are fit on calibration data by minimizing empirical cost.

Choosing the loss matrix is subjective, as the three-way decision literature acknowledges. Rather than defend one matrix, we will report results across a grid of cost ratios. We will also report one setting tied to an operational proxy, in which a field visit costs an order of magnitude more than an automated unmixing step.

A short analytical component will state when the band-group decomposition helps. With $n$ labels per class and $b = B/G$ bands per group, the envelope estimate in each group has to cover only a $b$-dimensional region. The number of samples needed to cover a region of fixed radius grows exponentially with dimension, which is the usual covering argument. This analysis predicts a crossover value of $G$ that shifts upward as $n$ shrinks, and we will test that prediction directly.

Baselines

The naive reference point is the same SpectralMamba backbone trained with cross-entropy. A pixel is deferred whenever its top softmax probability falls below a threshold, and deferred pixels are assigned to "mixed" or "unknown" by a second threshold on entropy. This is what a practitioner would do on day one. Beating it by a wide margin is necessary but not sufficient.

The strongest existing competitor is GPN-UR-TV from Uncertainty-aware Graph-based Hyperspectral Image Classification. It reports 94.55% out-of-distribution AUROC on Pavia University, against 82.82% for the plain graph posterior network. It is the only method in the file that already separates data uncertainty from distributional ignorance in hyperspectral images. To give it a fair three-way policy, we map its dissonance to defer-as-mixed and its vacuity to defer-as-unknown, and fit thresholds with the same calibration split and the same cost-minimizing procedure as ours. It gets the same training labels and endmember library, and a hyperparameter budget matched in trials. We first reproduce its single-class-holdout AUROC on Pavia. If our reproduction falls well short of 94.55%, we report both figures. Beating it means lower expected decision cost across the cost grid, and higher macro-F1 on routing among known, mixed and unknown pixels, at no worse out-of-distribution AUROC.

The third baseline is a TWDBDL-style multi-pass approach adapted from the skin-cancer three-way decision paper: five-member deep ensembles of the backbone with Monte Carlo dropout. Expected entropy routes pixels to unknown and mutual information routes them to mixed. It is compared at equal training compute, and inference latency is reported separately.

The key ablation is a band-group ensemble. It uses the same $G$ group readouts with ordinary softmax heads, averaged, and splits uncertainty into mutual information (disagreement) and expected entropy. This isolates whether independent contradiction evidence matters, or whether band grouping alone does the work.

Further ablations replace the neutrosophic degrees with coupled ones. One sets $F_k = 1 - T_k$, the Zadeh fuzzy version. The others project the degrees onto the intuitionistic constraint ($\mu+\nu\le 1$) and the Pythagorean constraint ($\mu^2+\nu^2\le 1$). These test the representation claim that the field has argued about only on toy tabular data.

Why not just…?

Why not just threshold the softmax? Per the graph-based paper, softmax training projects unseen materials onto known class directions with high confidence, which is why softmax GCN reaches only 57.04% AUROC on Pavia. More fundamentally, the softmax must place its mass somewhere, so "supports two classes" and "supports none" collapse to the same flat distribution. The naive baseline is included to measure exactly this. We expect it to be competitive on out-of-distribution AUROC for spectrally distant novelties and to fail on routing.

Why not just use the Dirichlet evidential model's dissonance and vacuity? This is the real threat. Evidential subjective logic already assigns belief to each class plus a leftover uncertainty mass, which is structurally close to an intuitionistic set. Our bet rests on two arguments. Vacuity is estimated from density over the full high-dimensional embedding, so it degrades with few labels. And dissonance cannot tell which spectral regions disagree, so a pixel that is ambiguous across the whole spectrum looks like a real two-material mixture. If GPN-UR-TV matches us on routing at 50 labels per class and at 5, we will report that the evidential split suffices and that our contribution reduces to the band-group design. The label-scarcity sweep settles this question.

Why not just ensemble classifiers over band subsets and read their disagreement? This is plausibly strong, which is why it is our main ablation and not merely a strawman. Disagreement among band groups already flags many mixtures. Where we expect it to fail is unknown materials. Each softmax head still confidently assigns the novel spectrum to some class, and if the heads agree by accident, mutual information stays low. Explicit envelope contradiction does not have this failure. If the ensemble matches us on unknown-material recall, the contradiction readout is not earning its place and we would drop it.

Why not just compress with PCA and use a standard classifier with rejection? SuperPCA shows that global PCA blurs local spectral differences. Global compression also merges the band regions whose disagreement is our mixture signal, so the method would lose the information it needs. We will include a global-PCA variant of the band-group model to show this directly.

Proposed Experiments

The experiments use Pavia University, Houston2013 and Indian Pines. Indian Pines has 16 imbalanced classes, heavy spectral overlap and coarse pixels, so many of its labeled pixels are genuinely mixed. Kennedy Space Center is added for comparability with the graph-based paper.

Ground truth for the three routing categories is built in three ways. Unknown materials come from holding out one class, and also from holding out two or three classes at once, to address the single-holdout limitation the graph-based paper leaves open. Mixed pixels come from two sources. The first is synthetic mixtures of held-out pure pixels at fractions between 0.3 and 0.7, using both linear and bilinear mixing, so that the method is not tested only under the linear assumption its competitor relies on. The second is natural pixels lying on class-map boundaries, which serve as a noisier real-world check. Pure known pixels are everything else.

Training labels per class are swept over 5, 10, 20 and 50, and $G$ is swept over 4, 8, 16 and 32. To look for the Hughes effect directly, we also subsample bands to 50%, 75% and 100%.

The metrics are expected decision cost across the cost grid, macro-F1 for routing among known, mixed and unknown, the risk–coverage curve (accuracy on accepted pixels against the fraction accepted), out-of-distribution AUROC for comparison with the 94.55% figure, and pixels per second on one GPU. For closed-set overall accuracy, the backbone should stay near the published reference points: 91.07% for SpectralFormer and 96.59% for DSFormer on Pavia, and 89.52% for SpectralMamba on Houston2013. If our model gives up more than about a point there, the uncertainty readouts are costing accuracy. No reported result was found on Lacuna for mixed-pixel deferral or three-way routing on any hyperspectral benchmark, so those numbers will be the first of their kind.

Four outcomes are decisive. First, routing macro-F1 must exceed GPN-UR-TV's dissonance/vacuity policy at matched coverage on at least two of the three main datasets. Second, the margin over both GPN-UR-TV and the band-group softmax ensemble must widen as labels per class fall from 50 to 5. Its absence would falsify the dimensionality argument. Third, the Zadeh and intuitionistic ablations must lose specifically on synthetic mixtures with strong support for both components, while the Pythagorean and neutrosophic variants keep them. If they lose everywhere or nowhere, the representation claim is unsupported. Fourth, the method must run in one pass at a speed comparable to the backbone, against at least an order-of-magnitude slowdown for the ensemble-plus-dropout policy.