Telling Atypical Specimens from Mislabeled Ones in Species Classifiers

When a species classifier disagrees with a training label, one of two things is usually true. The specimen may be correctly labeled but unusual: it was collected at an odd site, photographed against an odd background, or grew into an atypical form. Or the specimen may simply be mislabeled by the person who identified it. The two dominant families of training fixes give opposite prescriptions for the same signal. Debiasing methods upsample persistently disagreeing examples, because they treat them as the rare cases that break shortcuts. Noise-cleaning methods discard or relabel those same examples, because they treat them as annotator errors. In biodiversity data both causes are common and look alike: citizen-science and field identifications contain structured observer errors between look-alike species, and collections are geographically clustered. Applying either family blindly will either amplify errors or delete the diversity that out-of-distribution generalization depends on.

We propose to separate the two causes by asking which part of the input drives the disagreement. A context-only model sees the background, collection site and imaging metadata with the organism masked out. An organism-only model sees the cropped specimen. A sample that conflicts with context but agrees on organism appearance is a correctly labeled atypical specimen, and we upweight it. A sample whose organism appearance persistently contradicts its label is a label-error candidate, and we soft-correct it. When both models disagree, the sample is ambiguous and left alone. The testbed is BIOSCAN-5M, where every specimen has a DNA barcode. The barcode gives an identity that is independent of any human visual identification, so for the first time we can measure on real data whether a triage rule keeps the right samples and corrects the right ones.

Core Claim

Disagreement with a label is not a single quantity. It can be decomposed into a context-attributable part and an organism-attributable part. Using that decomposition to route samples should beat three things: pure debiasing, pure noise cleaning, and running the two in sequence. The advantage should show up in two ways at once. The method should detect real, DNA-verified identification errors more accurately, and it should keep more DNA-verified atypical specimens that are correctly labeled. Together these should give better worst-region accuracy under geographic shift.

Why the Obvious Pieces Conflict

Mitigating Spurious Correlations via Disagreement Probability (DPR) trains a deliberately biased teacher with a loss that makes it latch onto easy shortcuts. It then upsamples the samples where that teacher disagrees with the label. It assumes the labels are clean, so under observer error it upsamples mistakes. LiNC takes a different approach: it learns a trust weight per sample and sorts samples into clean, ambiguous and noisy groups. On Revisiting Entropy for Identifying Mislabeled Images (SEI) instead integrates a signed prediction entropy over training, which separates hard clean samples from mislabeled ones. Both LiNC and SEI were validated mainly on medical images, largely under synthetic or symmetric noise, and neither asks whether the "hard" part of a sample is the organism or its surroundings. These three papers supply the methods we reuse and compete against. We borrow DPR's biased-model idea and SEI's use of training trajectories, but we apply each separately to the context and organism views instead of to the whole image.

Three papers motivate the proposal by documenting the two sources of disagreement. Open-Insect benchmarks detection of novel moth species across regions. It shows that open-set detection collapses when insect features are masked, which means classifiers lean heavily on background, and that geographically close novel species are hardest to catch. BIOSCAN-5M pairs 5.15 million insect images with DNA barcodes. It shows image clustering agrees with species identity only 21% of the time, against over 90% for DNA, so visually driven identification errors between look-alike species are expected and can be checked. BioCLIP trains an image–text model on 10.4 million organism images labeled with full taxonomic names. It acknowledges taxonomic noise across its sources but does not model it.

Four papers constrain the design. The citizen-science work of Hines et al., which weights each volunteer's vote by that volunteer's reliability on the species in question, and the class-conditional species-distribution model, which models false sightings from observer effort metadata, both show that observer error is species-specific and structured, not uniform. We therefore generate structured, not symmetric, noise. Wilm et al. study mitosis detection trained on labels from individual pathologists versus consensus groups. They find model F1 plateaus near 0.74–0.76 however clean the labels are. That warns us that cleaner labels alone may not move accuracy, so we measure detection quality and retention directly, not only final accuracy.

Research Program

Reference identity. On BIOSCAN-5M we assign each specimen a reference identity from its DNA barcode, using nearest-neighbor consensus among barcoded specimens embedded by a DNA model of the kind benchmarked in BarcodeMamba. Taxonomic names that conflict with this identity become candidate real label errors. Because we cannot assume how many such conflicts exist, we also build a controlled noise channel that mimics observers. Among DNA-distinct species that an image model confuses, typically congeners, we flip labels in proportion to the image model's confusion rates. The result is noise concentrated on look-alike pairs, as in citizen science.

Atypical-but-correct specimens. We define these as samples whose DNA identity matches the label but whose collection region, site or imaging batch is rare for that species. Every sample therefore carries a verifiable category: clean-typical, clean-atypical, or mislabeled.

Triage method. We train two early probes on top of a frozen BioCLIP backbone. The context probe sees the image with the organism segmented out, plus coordinates and imaging metadata. The organism probe sees only the crop. For each sample and each probe we record the signed-entropy trajectory over training, following SEI. The routing rule works as follows. If the context trajectory is persistently negative but the organism trajectory turns positive, the sample receives DPR-style upsampling. If the organism trajectory stays negative, the sample gets a LiNC-style trust weight that shrinks its target toward the model's prediction. The thresholds are fit on a small DNA-audited calibration split of a few thousand specimens, never on the test split.

Transfer without DNA. For deployment where no DNA exists, we transfer the rule to Open-Insect. The moth images there have real field backgrounds, and we inject the same kind of confusion-structured noise among visually similar local species.

Baselines

Plain training (the naive reference). Standard cross-entropy on the observed labels with the same frozen-then-fine-tuned BioCLIP backbone, the same augmentations, and the same epoch budget. Every other method must beat it on both error detection and worst-region accuracy to count.

DPR Mitigating Spurious Correlations via Disagreement Probability. This is the strongest debiasing competitor because it is the only method in the file that finds bias-conflicting samples without bias labels. The file reports a 20.87% gain in minority-group accuracy over plain training on Biased FFHQ, a face dataset with strong spurious correlations. Its reported result is on faces, not species data.

LiNC LiNC. This is the strongest noise-cleaning competitor. It reports a noise-detection AUC of 0.9837 on OrganSMNIST, a small organ-image dataset, at 20% noise.

SEI On Revisiting Entropy for Identifying Mislabeled Images. SEI reports over 8% F1 improvement over Cleanlab, AUM and CORES on the ISIC 2018 skin-lesion dataset at 50% noise.

For BIOSCAN-5M and Open-Insect, no reported result was found on Lacuna for noise detection or debiasing by any of these methods, so we run all three ourselves. Each baseline gets its authors' default hyperparameters plus the same random-search budget of 20 configurations, tuned on the same calibration split our method uses.

Sequential combination. SEI filtering followed by DPR on the survivors. This is the most natural composite that lacks our key ingredient.

Single-view ablation. Our routing rule with both probes replaced by one whole-image probe. This isolates whether splitting the input into context and organism views is what does the work.

DNA-label oracle. Training on DNA-derived identities. This is an upper bound, not a competitor.

Our method beats the field if it matches the best baseline's AUC for detecting mislabeled samples while retaining clearly more clean-atypical samples, and if this translates into higher accuracy on the held-out region.

Why not just…?

Why not just relabel everything with DNA? Where barcodes exist, one should. The oracle row shows how much that is worth. But camera traps, light traps and platforms like iNaturalist produce image-only data. BIOSCAN-5M is valuable here precisely because we can hide the DNA from the method and use it only for scoring.

Why not just crop the organism and train an organism-only classifier with SEI cleaning? This is a real threat. Cropping removes much of the background shortcut, and SEI then handles noise. Open-Insect's masking result shows background matters, but it does not show that cropping plus cleaning is insufficient. We run this baseline directly. If it matches our method on retention and worst-region accuracy, the decomposition is unnecessary and we will report that. Our expectation is that it fails in two ways. Site-driven differences in the organism itself, such as body size or coloration, remain in the crop. And metadata shortcuts such as imaging batch are invisible to a crop-only model.

Why not just clean labels using nearest-neighbor votes in BioCLIP embedding space? Neighbor voting is cheap and strong when the embedding separates species. The 21% species-level image clustering agreement in BIOSCAN-5M suggests it will fail on exactly the look-alike pairs where observers also fail. That failure would repeat the systematic-bias problem that LiNC's authors flag. We include a nearest-neighbor cleaner as a baseline and report its results separately for look-alike pairs.

Why not just run SEI and then DPR? That is the sequential baseline. The SEI stage should delete clean-atypical samples before DPR ever sees them, which is Gap 1 in miniature. The DNA-verified retention metric will show whether this happens.

Proposed Experiments and Decisive Checks

The first check is diagnostic and must precede any method claim. On BIOSCAN-5M with DNA-verified categories, we measure two quantities for DPR, LiNC and SEI: what fraction of clean-atypical samples each discards or down-weights, and what fraction of mislabeled samples each upsamples. If the contradiction does not appear, meaning noise cleaners keep atypical samples and DPR ignores errors, the premise fails and we stop.

The second check compares all methods on three measures. The first is AUC for detecting mislabeled samples, computed separately for real DNA conflicts and for the injected look-alike noise at 10%, 20% and 40% rates. The second is retention of clean-atypical samples. The third is species accuracy on a DNA-verified test set drawn from a held-out collection region. We take the worst region as the primary number, because Open-Insect shows close-region shift is the hard case.

The third check is transfer to Open-Insect with injected look-alike noise. There we report closed-set accuracy and open-set detection AUROC. The file reports AUROC above 85% for simple maximum-softmax detection on local novel species. The question is whether uncorrected label noise erodes that figure, and whether our routing recovers it better than the baselines.

The decisive ablation is the single-view version. If it performs within noise of the two-view method, the claim that attributing disagreement to context versus organism is what matters is refuted.