Geometric Distance as a Proxy for Model Uncertainty

71 concepts53 papers1 proposal

When a machine learning model makes a prediction, how do we know we can trust it? The standard approach is to look at the final layer's softmax probabilities. If a model outputs a 99% probability for a class, we treat it as highly confident. However, softmax is notoriously prone to overconfidence and saturation. Because it normalizes outputs into a probability distribution, it discards the underlying spatial layout of the data. A sample sitting right on the edge of a decision boundary can receive the same high-probability score as a sample buried deep within a well-defined cluster.

To build more honest models, a growing body of research shifts the focus from probability space to geometric space. The core intuition is simple: distance is a natural proxy for uncertainty. By measuring how far an input is from a decision boundary, a class prototype, or the training data itself, we can derive continuous, stable confidence scores that reflect the model's true spatial certainty.

The Proximity-of-Errors Hypothesis

Why does geometry map so cleanly to uncertainty? The foundational assumption is the proximity-of-errors hypothesis: classification errors are not distributed randomly; they cluster heavily near decision boundaries. As Alvarez et al. observe, the closer a data point is to a model's split or decision boundary, the more prone it is to misclassification.

By calculating the algebraic or geometric distance from an input to this boundary, we can transform a discrete classification into a smooth, continuous confidence metric. This geometric margin has been successfully leveraged across diverse domains. For instance, Suzuki et al. repurposed the hyperplane margin of Support Vector Machines to rank open-domain question-answering candidates, while Karakoulas et al. used it to optimize boundaries for highly imbalanced datasets. Whether filtering high-stakes text streams (Mishler et al.) or predicting personalized cancer prognoses (Shao et al.), tracking proximity to the decision boundary provides a reliable buffer against false positives.

Measuring Distance via Prototypes and Counterfactuals

In modern deep neural networks, explicit decision hyperplanes can be incredibly complex and difficult to compute directly. To circumvent this, researchers project data into learned embedding spaces and measure geometric relationships relative to reference points.

One prominent strategy uses class prototypes—the central representations or centroids of each class in the feature space. Instead of relying on softmax outputs, Xing et al. demonstrate that the $L_2$ distance between an input embedding and its corresponding class prototype provides a highly monotonic and stable calibration signal. This geometric relationship ensures that as a sample becomes more recognizable, its feature magnitude naturally moves further from the boundary (Pang et al.).

To make this metric robust against overlapping classes, Schleif et al. and Hammer et al. calculate a relative margin: the ratio or difference between the distance to the nearest correct prototype and the nearest incorrect prototype. This spatial comparison allows safety-critical systems, such as pedestrian detectors, to ground their decisions in quantifiable geometric proximity rather than opaque non-linear transformations (Feifel et al.).

Another elegant way to locate the decision boundary is through counterfactual examples. A counterfactual represents the smallest possible feature perturbation required to flip a model's prediction. Punzi et al. show that measuring the exact distance to the nearest counterfactual serves as an intuitive proxy for uncertainty. If an instance requires only a tiny spatial nudge to change its predicted class, it resides dangerously close to the decision boundary and should be flagged for rejection. Similarly, Yue et al. utilize the number of adversarial steps needed to cross a boundary as a granular metric for class difficulty.

Model-Agnostic and Instance-Level Uncertainty

One of the greatest strengths of geometric uncertainty estimation is that it can operate independently of the underlying model architecture. By focusing on the intrinsic geometry of the feature space, we can detect out-of-distribution or ambiguous samples without modifying neural network backbones.

For example, Chouraqui et al. and Nan et al. show that the simple Euclidean distance between a query point and its nearest neighbors in the training set or a reference database acts as an exceptional baseline for uncertainty.

In active learning, where the goal is to select the most informative unlabeled samples to present to human annotators, Guo demonstrates that sampling based purely on the geometric structure of the feature space decouples data selection from specific classification models. Similarly, Wen et al. show that focusing on feature-space diversity ensures compatibility across diverse neural network backbones. This geometric grounding prevents the model from falling into overconfidence traps during semi-supervised pseudo-labeling (e.g., TK-KNN).

Why Softmax Fails Where Geometry Succeeds

To understand why geometric approaches are so effective, it helps to look at how standard probability normalization distorts feature space. When a model computes a softmax distribution, it normalizes logits across classes, which completely discards the absolute scale or "energy" of the features.

As Attar et al. point out, energy-based metrics can recover this lost geometric information, allowing models to distinguish between samples that share identical softmax probabilities but sit at vastly different distances from the actual decision frontier. Furthermore, the spatial arrangement of prototypes themselves dictates confidence limits; Fillioux demonstrates that the geometric distance between text prototypes in vision-language models sets a mathematical lower bound on softmax confidence.

When boundaries are thin, models become fragile. By measuring "boundary thickness"—the average distance across a classifier's uncertainty zone—the authors of Boundary thickness and robustness in learning models show that a wider geometric buffer is a far better predictor of robust generalization than worst-case margins.

Key Foundational Work

Optimizing Classifers for Imbalanced Training Sets (1998) — contributed 1 concept elements to this direction.
SVM Answer Selection for Open-Domain Question Answering (2002) — contributed 1 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_geometric_uncertainty_estimation_f0bbdfa8,
  title = {geometric uncertainty estimation},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/geometric-uncertainty-estimation/rdr_f0bbdfa825454e13a3b80874dcbfe02d},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_geometric_uncertainty_estimation_f0bbdfa8_v1,
  title = {geometric uncertainty estimation},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/geometric-uncertainty-estimation/rdr_f0bbdfa825454e13a3b80874dcbfe02d/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake