Airborne tree-species maps are increasingly used in place of ground surveys, which are slow and expensive. In these surveys crews walk plots, tag stems and identify species by hand, so they rarely cover more than a small area. The maps fail most on rare species, and rare species are often the ones ecologists care about most. We propose to test a specific and checkable explanation for that failure. A canopy classifier may assign a rare label to a crown because the crown does not look like any common species, not because it shows the rare species' own spectral or structural signature. If so, the model is acting as an "absence filter," and its rare-species predictions will be wrong in a systematic and predictable way. Any unusual crown, whether a species missing from training, a stressed tree, a shadowed crown or a crown mixed with its neighbours, will be pushed into a rare class.
The proposal has two parts. First, we measure how much of each rare-class prediction rests on suppressed competitors rather than positive evidence. We do this with a held-out-species behavioral probe and with a decomposition of the model's decision margin that borrows from negative-evidence explanation methods. Second, we use the per-crown measure to decide which aerial predictions a field crew should verify on the ground. Ground visits are the scarce resource. A per-crown score that says "this rare label is held up by what is missing" is useful only if it catches mistakes that ordinary model confidence misses, and we test exactly that.
Core Claim
Work on negative evidence has shown that deep vision models learn "encoded absences." What is Missing? Explaining Neurons Activated by Absent Concepts found that 2,036 of 2,048 final-layer ResNet-50 channels are shaped by inhibitory inputs. In the same paper, inserting absence-encoding patches flipped ImageNet predictions 84% of the time, against 8% for random patches. Certified Evaluation of Model-Level Explanations for Graph Neural Networks showed graph classifiers labelling a class purely because other classes' motifs were missing. Remote-sensing species models have the conditions that should make this shortcut attractive: a few dominant species supply most of the training crowns, and rare species have tens of samples. The OSBS hierarchical model in Capturing long-tailed individual tree diversity using an airborne multi-temporal hierarchical model reaches 96% precision and recall on Pinus palustris but only 64% macro-accuracy. Continental-scale hyperspectral tree species classification in the United States National Ecological Observatory Network reports sharp drops below 50 samples.
Our claim has three parts. Rare-class decisions in these models are disproportionately driven by competitor suppression. This reliance can be measured per crown. The per-crown measure predicts misclassification beyond what calibrated confidence already predicts. The third part matters most, because it turns an explanation method into a tool for allocating field work. Nobody in the file has tested any of the three. The two papers that connect explainability and environmental observation stop short of this question. An earth observation and explainable machine learning approach for determining the drivers of invasive species — a water hyacinth case study applies additive SHAP values to tabular catchment variables. SatBird handles impossible species with an external range mask rather than examining the model's own reasoning.
Research Program
Behavioral probe. The first component needs no explanation method at all. We retrain each classifier with one species withheld entirely, then feed it that species' crowns. A model that has learned positive signatures should spread these crowns over their nearest look-alikes, which are usually congeners or species with similar structure. An absence filter should send them disproportionately to rare classes, because rare classes are where "not any common species" lives in the decision space. We define the false-rare rate as the fraction of withheld-species crowns labelled with a rare class. We compare it with the share of training data that rare classes hold, and with the confusion pattern expected from spectral nearest neighbours. Repeating this over every withheld species gives a model-level absence-reliance estimate.
Per-crown margin decomposition. The second component works per crown. It follows the observation in "Why Not Other Classes?": Towards Class-Contrastive Back-Propagation Explanations that explaining raw scores reveals only "objectness," whereas class discrimination lives in the softmax, where a class wins either by rising or by its competitors falling. For a crown $x$ predicted as class $t$ with logits $z(x)$, let $m(x) = z_t(x) - \log\sum_{s\neq t} e^{z_s(x)}$ be the log-odds margin. We compare $m(x)$ with the median margin $\bar m_t$ of correctly classified class-$t$ training crowns, and split the excess into an own-evidence term and a competitor-suppression term:
$$m(x)-\bar m_t = \underbrace{\big(z_t(x)-\bar z_t\big)}_{\text{own evidence}} \;-\; \underbrace{\Big(\log\textstyle\sum_{s\neq t} e^{z_s(x)} - \overline{\log\textstyle\sum_{s\neq t} e^{z_s}}\Big)}_{\text{competitor suppression}}.$$
Here bars denote the class-$t$ reference medians. The absence-reliance score $A(x)$ is the competitor-suppression share of the total absolute change. $A(x)$ is nearly free to compute, which matters at the scale of the 104,675,304 crowns in NEON Crowns. For crowns flagged as high-$A$, we then localize the inhibitory evidence: which wavelengths, pixels or height strata push competitors down. For this we use the negative-factor decomposition of Generating Attribution Maps with Disentangled Masked Backpropagation, extended from three RGB channels to hyperspectral bands, and the weighted contrastive backpropagation of Wang et al. The localization step is diagnostic. The triage claim rests on $A(x)$ alone, and an ablation checks whether localization adds predictive value.
Counterfactual edits. Explanations of absence require a realistic "null" input. Explanations based on the Missing uses a learned autoencoder for this, and setting hyperspectral bands or LiDAR returns to zero creates physically impossible inputs. We therefore build counterfactuals only by transplanting real data. We blend the spectra of the bands identified as inhibitory toward a real crown of the suppressed dominant species, matched for canopy height and illumination. If a high-$A$ rare prediction flips under a small, physically real edit of these bands, while the rare species' own diagnostic bands are untouched, that confirms the decomposition.
Fix. The third component is a repair attempt. We add "presence plus absence" debiasing in the spirit of Hesse et al., which on ISIC raised inverted-data accuracy from 0.77 to 0.81. We also add an explicit outlier class trained on crowns from withheld or unlabeled species. We then ask whether macro-accuracy and the false-rare rate improve together.
Baselines
The naive reference point is uniform random selection of predicted-rare crowns for field verification. Paired with it is the unmodified classifier's macro-accuracy on each benchmark. Every triage method must beat random selection to be worth anything.
The strongest competitor for the triage question is calibrated predictive confidence: the softmax probability of the predicted class, plus its entropy and margin variants. Marconi et al. report that their ensemble's confidence is well calibrated (calibration R² = 0.89 on the NEON 27-site data). Confidence is free, so it is the method a practitioner would actually use. We compute it on exactly the same classifiers, crowns and held-out splits as $A(x)$. We also include an equal-budget combination of confidence and $A(x)$, so the question is whether $A(x)$ adds information, not whether it replaces confidence. Beating this baseline means a statistically significant increase in errors caught per fixed verification budget (for example the 5%, 10% and 20% least-trusted rare predictions), or a significant gain in error-detection AUROC when $A(x)$ is added to confidence, across sites and seeds.
The strongest existing classifiers serve as the substrate and as accuracy baselines. These are the OSBS multi-temporal hierarchical model (75% micro-accuracy, 64% macro-accuracy, rare-species accuracy 0.63 versus 0.48 for the baseline CNN), the Marconi pooled ensemble (micro-F1 0.77), and on BCI drone imagery the crown-view classifier from Understanding Representation Gaps across Scales in Tropical Tree Species Classification from Drone Imagery (74.3% Top-1). We reimplement each where code permits and report our reproduced numbers next to the published ones. For the repair step, success means lowering the false-rare rate without dropping macro-accuracy below the reproduced baseline under the same tuning budget. No reported result found on Lacuna for held-out-species false-rare rates on any of these benchmarks, so we establish that number ourselves.
Two ablations isolate the key ingredient. The first replaces the competitor-suppression share with a non-contrastive saliency score, Grad-CAM-style magnitude on the predicted logit alone. This tests whether the contrastive split, rather than any attribution, carries the signal. The second adds the DMBP localization features to $A(x)$ to test whether spatial and spectral localization adds predictive value beyond the logit-level decomposition.
Why not just…?
Why not just use confidence to decide which predictions to verify? This is the real threat, and the experiment is built to settle it. The hypothesis is that absence-filter predictions can be confident: a crown that strongly suppresses every common class can yield a high rare-class probability with little positive evidence. Calibration is measured on in-distribution crowns, so its R² of 0.89 says nothing about withheld species. If $A(x)$ adds no detection power beyond confidence on held-out-species and rare-class errors, we will report that the absence mechanism exists but has no operational value. That would be a negative result worth publishing.
Why not just rely on the hierarchical taxonomic model? The OSBS hierarchy lifts rare-species accuracy from 0.48 to 0.63, so it clearly helps. But a hierarchy can just as easily route "not a pine" to a rare hardwood genus, so it does not rule out the shortcut. We apply the held-out-species probe to the hierarchical model directly. If its false-rare rate stays at its base-rate share, the hierarchy already cures the problem and we will say so.
Why not just run the held-out-species test and skip attribution? The behavioral probe is the cleanest model-level evidence and leads the program for that reason. It cannot say which individual crowns in a 670,883-crown census are suspect, and field triage needs that. The per-crown score earns its place only if it beats confidence in the triage experiment, so the two are complementary rather than redundant.
Why not just add LiDAR or more spectral bands? Fusion raises overall accuracy (81.0% weighted F1 for LiDAR versus 74.6% for imagery in PureForest). Marconi et al. still find rare-species failure after fusing hyperspectral and LiDAR data. We run the probe on single-modality and fused variants of the same architecture. If fusion alone drives absence reliance down, that becomes the practical recommendation.
Proposed Experiments
Data. The primary testbed is the OSBS NEON survey: 14 species, 1,556 field samples, four years of hyperspectral, RGB and LiDAR data. We use it because rare taxa below 1% landscape frequency are labelled there. The NEON 27-site tree dataset (77 species) tests whether absence reliance grows as the species pool and the imbalance grow. The BCI tropical drone dataset (84 species) tests a high-diversity canopy, and its paired close-up imagery adds a further check: crowns whose rare label is high-$A$ in the crown view should more often be relabelled when close-up botanical detail is available.
Diagnosis check. The first decisive check runs the probe across all withheld species on all three datasets. It asks whether the false-rare rate exceeds both the rare-class training share and a nearest-neighbour spectral confusion baseline, and whether the median $A(x)$ for correctly classified rare crowns is significantly higher than for common crowns. If neither holds, the hypothesis is false and the remaining program stops.
Triage check. The second decisive check simulates field verification on held-out labelled crowns. We rank predicted-rare crowns by random order, by confidence, by $A(x)$, and by both combined, and count true errors caught per verification budget. The added AUROC from $A(x)$ is the headline number.
Edit and repair checks. The third check applies the transplant edits to high-$A$ and low-$A$ crowns and compares flip rates. Following the Hesse et al. protocol, high-$A$ predictions should flip far more often under edits to inhibitory bands than low-$A$ predictions do under matched edits. The fourth check measures whether absence-aware debiasing and an outlier class reduce the false-rare rate while holding macro-accuracy. Throughout, we report site-level variance, because spatial autocorrelation between training and test crowns can inflate every one of these numbers.
References
- What is Missing? Explaining Neurons Activated by Absent Concepts
- Certified Evaluation of Model-Level Explanations for Graph Neural Networks
- Capturing long-tailed individual tree diversity using an airborne multi-temporal hierarchical model
- Continental-scale hyperspectral tree species classification in the United States National Ecological Observatory Network
- An earth observation and explainable machine learning approach for determining the drivers of invasive species — a water hyacinth case study
- SatBird: a Dataset for Bird Species Distribution Modeling using Remote Sensing and Citizen Science Data
- "Why Not Other Classes?": Towards Class-Contrastive Back-Propagation Explanations
- NEON Crowns: a remote sensing derived dataset of 100 million individual tree crowns
- Generating Attribution Maps with Disentangled Masked Backpropagation
- Explanations based on the Missing: Towards Contrastive Explanations with Pertinent Negatives
- Understanding Representation Gaps across Scales in Tropical Tree Species Classification from Drone Imagery
- PureForest: A Large-Scale Aerial Lidar and Aerial Imagery Dataset for Tree Species Classification in Monospecific Forests