Where Confident Mistakes Hide: Low-Variance, Curved Feature Geometry for Selective Classification

A selective classifier is allowed to say "I don't know." It pairs a predictor with a gate that abstains on risky inputs, so that the answers it does give are almost always right. Most gates read the model's own confidence, for example the top softmax probability or the gap between the top two logits. A second family scores how typical an input's internal features look, for example its Mahalanobis distance to its class cluster. This note argues that both families share one blind spot. Neural networks put most of their feature variance into simple, linearly readable cues. The harder cues that actually decide difficult cases sit in low-variance directions, and those directions are arranged non-linearly. When an input matches the easy cues but is ambiguous on the hard ones, the network is confident, variance-dominated distances call it typical, and a single class ellipsoid cannot see the non-linear arrangement. The mistake passes through the gate.

We propose a selection score built to look where these mistakes live. First we whiten the feature space, so that low-variance directions are no longer drowned out. Then, instead of fitting one global ellipsoid per class, we fit a small local plane to the predicted class's nearest training neighbours. The score is how far the input lies off that local plane. The contribution is a testable claim with a clean two-by-two ablation: catching confident errors requires both removing variance dominance and modelling curvature locally, and neither alone is enough. We test the claim first on a synthetic task where the answer is known, then on standard image benchmarks with and without distribution shift.

Core Claim

Two results motivate the claim.

The first concerns what large-variance directions encode. Representation Biases: Variance Is Not Always a Good Proxy for Importance trains networks on tasks that need both an easy linear feature and a hard parity (XOR-like) feature. In the penultimate layer, the easy feature takes over 50% of the variance and the equally necessary hard feature takes about 5%. That paper uses the finding only as a warning for interpretability and never turns it into an abstention rule. We do exactly that.

The second concerns curvature. Classic work on linear dimensionality reduction shows that a flat projection misreads bending as extra dimensions and places far-apart points next to each other (Non-Linear Dimensionality Reduction; Nonlinear Dimension Reduction via Local Tangent Space Alignment). A parity feature is an extreme case of this. Within one class it produces many separate clusters, so a single ellipsoid fitted to the class covers the empty space between them.

Our claim follows from the combination. Among errors made at high confidence, a large share are inputs that sit off the local class structure in the whitened, low-variance part of feature space. Such errors are caught by a score that is both whitened and local, and missed by scores that are only one of the two.

Concretely, let $z$ be the penultimate feature of input $x$. Let $\hat y$ be the predicted class. Let $u = \Sigma^{-1/2}(z-\mu)$ be the feature after whitening with the global training mean $\mu$ and a shrinkage-regularised covariance $\Sigma$. Let $N_k$ be the $k$ nearest training points of class $\hat y$ to $u$, with mean $m$ and top-$d$ principal basis $U$. The score is

$$ s(x) = \frac{\lVert (I - UU^\top)(u - m)\rVert^2}{\bar r_k}, $$

where $\bar r_k$ is the average of the same off-plane residual computed for the neighbours themselves. Dividing by $\bar r_k$ makes the score relative to how noisy that region is, so it does not depend on local scale. This is a local tangent-plane residual in the spirit of Fast Non-Linear Dimension Reduction and LTSA. Here it is used as a rejection signal rather than to build an embedding.

Research Program

The work has three parts.

A controlled testbed. We rebuild the easy-plus-parity task of Representation Biases, where we know exactly which feature decides each label. We then plant "conflict" test inputs, whose easy cue points to one label while the parity cue determines the true one. Here the claim can be checked directly: does each score reject the conflict errors, and does its behaviour match the predicted two-by-two pattern?

A measurement on real networks. Before believing the mechanism transfers, we ask where errors can be told apart from correct predictions. For trained ResNets and pretrained encoders, we split feature space into leading and trailing principal subspaces. In each, we measure how well a simple local score separates the errors that a strong confidence score accepts. If the accepted errors are only distinguishable in the leading subspace, the premise fails for real networks, and we will report that as the finding.

The method under certified thresholds. The score is dropped into the risk-guaranteed thresholding procedure of Selective Classification for Deep Neural Networks. That procedure takes a user-chosen error bound and a confidence level and calibrates a threshold on held-out data, so every score is compared under the same statistical guarantee. We also test a simple combination: a logistic rule fitted on validation data that merges $s(x)$ with the confidence margin. Every baseline receives the same combination.

Baselines

The naive reference point is Softmax Response, the top softmax probability, on a standard cross-entropy model. Selective Classification for Deep Neural Networks reports a 1% error rate at 78% coverage on CIFAR-10 with this signal under its guarantee procedure, and a 2% top-5 error at about 60% coverage on ImageNet. We reproduce these operating points first, to anchor our pipeline.

The strongest confidence-based competitor is Softmax Response on a model trained with self-adaptive training plus entropy minimisation. Towards Better Selective Classification reports up to an 80% relative improvement over earlier selective methods on ImageNet subsets with this recipe. Next to it we run the scale-invariant confidence margin and geometric margin of Selective Classification Under Distribution Shifts. That paper reports them as best overall when all shift types are present, but no numeric value for them was found on Lacuna. All confidence scores are computed from the same trained backbone our score uses, so only the gate differs. Our score must beat this group on the area under the risk-coverage curve (AURC, the average error over all coverage levels), or on coverage at a fixed certified risk, or add to it measurably in the combined rule. Otherwise the proposal fails.

The strongest geometric competitor is relative Mahalanobis distance, following Learning Sample Difficulty from Pre-trained Models for Reliable Prediction. That paper uses the score to reweight training, reporting +0.55% accuracy and a 3.7% ECE reduction on ImageNet-1k with ResNet34. No direct selective-risk number for it as a gate was found on Lacuna. We use it directly as a gate on the same features.

We also run the learned selector of Calibrated Selective Classification, which feeds entropy, kernel density and Isolation Forest scores to a small network. We run it twice, with and without $s(x)$ as an extra input. That paper's reported gain is an 18–25% reduction in calibration-error AUC on corrupted benchmarks.

The key ablation is the two-by-two grid:

  1. raw-space global distance, which is plain Euclidean distance to the class mean;
  2. whitened global distance, which is Mahalanobis distance;
  3. raw-space local residual;
  4. whitened local residual, which is ours.

We add raw and whitened kNN distance as a no-plane control. All geometric scores share the same features, neighbour budget, and validation-tuned choices of $k$, $d$ and shrinkage, tuned with equal search effort.

Why not just…?

Why not just use confidence margin after entropy-minimised training? This is the real threat. Towards Better Selective Classification shows that simple confidence scores beat specialised selection heads, and our score may add nothing. The decisive measurement is complementarity. We take the errors accepted by the margin at 90% coverage and ask what fraction our score flags, compared with the fraction among correct accepted predictions. If that difference is negligible on real data, the geometric view contributes nothing beyond confidence.

Why not just use Mahalanobis distance, which already whitens? Mahalanobis distance does remove variance dominance. But one ellipsoid per class cannot represent the multi-cluster geometry that non-linear features create. This is the curvature failure documented by Non-Linear Dimensionality Reduction, and the file flags it as unaddressed by the difficulty-scoring work. The whitened-global versus whitened-local cell of the ablation tests this directly. If they tie on real benchmarks, locality is not earning its cost.

Why not just use kNN distance, which is local and simpler? It might work as well. The only extra ingredient we add over kNN is the local plane, and whitened kNN is in the ablation for exactly this reason. If whitened kNN matches the plane residual, we will report the simpler score as the method. The central claim, that a score must be both whitened and local, would still stand.

Why not just project onto the top principal components to denoise before scoring? On the evidence of Representation Biases, this discards the roughly 5%-variance directions where the decisive hard features live. The raw-space cells of the ablation, together with a top-principal-component variant, show whether this happens in practice.

Proposed Experiments

Synthetic testbed. The metric is the rejection rate on conflict errors at matched coverage. The predicted pattern is that only the whitened local score rejects them at high rates. Raw scores should fail because variance dominance hides the parity directions. Mahalanobis distance should fail because the parity clusters fill its ellipsoid.

Standard benchmarks. We use CIFAR-10, CIFAR-100, ImageNet subsets, StanfordCars and Food101, as in Towards Better Selective Classification. We report AURC, coverage at 1% and 2% certified risk under the guarantee procedure, and the complementarity statistic.

Shift benchmarks. We use CIFAR-10-C, ImageNet-C and iWildCam, following the generalized setting of Selective Classification Under Distribution Shifts, and track how each score's advantage changes as corruption severity rises.

Decisive checks. There are two, and they tell us which way the proposal turned out. The first is whether, on real networks, confidence-accepted errors are separable mainly in trailing principal subspaces. The second is whether the whitened local cell beats both whitened-global and whitened-kNN on at least the fine-grained and shifted benchmarks, after the tuning budget has been equalised.