Local Prediction Stability for Uncertainty Estimation

48 concepts35 papers

Deep neural networks are notoriously overconfident. As highlighted by Susmit Jha et al. in Attribution-Based Confidence Metric For Deep Neural Networks, a model can be 99% certain about an adversarial or out-of-distribution input that is completely incorrect. This occurs because standard confidence metrics, such as softmax probabilities, only reflect a single point's position relative to a decision boundary, ignoring the local topology of the decision space.

To build an intuition for why this happens, imagine the model's decision landscape. A correct, robust prediction typically resides deep within a wide, homogeneous valley of a single class. In contrast, an incorrect or adversarial prediction often sits precariously on a sharp ridge or right next to a volatile decision boundary. While both points might yield identical softmax scores, their local neighborhoods look entirely different. By shifting our focus from single-point evaluations to local prediction stability, we can probe this geometry to discover how reliable a prediction truly is.

Probing the Neighborhood via Input Perturbations

The most direct way to test local stability is to apply small, stochastic perturbations to the input and observe whether the model's prediction flips. If a model's classification collapses under minor changes, the prediction is brittle.

The authors of DOCTOR: A Simple Method for Detecting Misclassification Errors demonstrate that this acts as a diagnostic stress test: correct classifications remain stable under small input modifications, whereas incorrect but overconfident predictions quickly fall apart. This principle generalizes across diverse modalities:

By aggregating these local responses, it becomes possible to predict broader model behavior. For instance, Nathan H. Ng et al. propose Neighborhood Invariance in Predicting Out-of-Domain Generalization with Neighborhood Invariance, calculating the frequency of the most common predicted class within a neighborhood of transformed variants to accurately forecast out-of-domain generalization without any labels.

Shaking the Model: Internal and Weight Perturbations

Instead of moving the input data point, a similar diagnostic effect can be achieved by slightly perturbing the model itself. If a minor internal fluctuation drastically alters the output, the model lacks a stable representation of that specific input.

Li Dong et al. utilize this in Confidence Modeling for Neural Semantic Parsing by applying dropout or weight noise to identify low-confidence semantic parses. Similarly, Matteo Ferrante demonstrates in Enabling uncertainty estimation in neural networks through weight perturbation for improved Alzheimer's disease classification that post-training weight perturbations in the final layer act as a direct probe of the local loss landscape. If the output remains stable across multiple perturbed versions of the weights, the prediction is robust.

This internal probing can also be applied at the logit or state level. Linwei Tao et al. develop Consistency Calibration: Improving Uncertainty Calibration via Consistency among Perturbed Neighbors, replacing standard softmax probabilities with the fraction of local neighbors that maintain the same class under logit-level perturbations. In the context of large language models, Pragatheeswaran Vipulanandan et al. show in Semantic Uncertainty Quantification of Hallucinations in LLMs: A Quantum Tensor Network Based Method that applying small mathematical perturbations to internal states can successfully isolate volatile, hallucination-prone regions from stable, trustworthy outputs.

Prompt and Context Sensitivity in Language Models

In-context learning (ICL) introduces another dimension of instability: prompt phrasing. While a model's sensitivity to prompt variations is usually seen as a limitation, Yanda Chen turns this into a feature in On the Relation between Sensitivity and Accuracy in In-Context Learning. By measuring output stability across multiple paraphrased or reordered variations of the same prompt, the proposed SENSEL method allows the model to selectively abstain from making a prediction when its local sensitivity signals a high likelihood of error.

Leveraging Stability for Learning and Privacy

Because local prediction stability strongly correlates with correctness, it serves as a powerful filtering mechanism in semi-supervised learning, where models must learn from unlabeled data without accumulating errors.

Interestingly, this exact relationship can be inverted for privacy attacks. In Chameleon: Increasing Label-Only Membership Leakage with Adaptive Poisoning, Harsh Chaudhari et al. demonstrate that membership status in label-only models is highly visible through neighborhood stability. Because models naturally overfit to their training data, a target sample that was part of the training set will display exceptionally high classification stability across its local neighborhood compared to an unseen sample, creating a significant privacy vulnerability that Harsh Chaudhari further explores in Chameleon: Increasing Label-Only Membership Leakage with Adaptive Poisoning.

Key Foundational Work

An Ensemble Method for Selection of High Quality Parses (2007) — contributed 1 concept elements to this direction.
Hallucination-Free Multi-View Stereo (2010) — contributed 1 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_local_prediction_stability_4c28110c,
  title = {local prediction stability},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/local-prediction-stability/rdr_4c28110c363f4b31a45be88ef0231f16},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-19},
}

This exact version (version 1, frozen text):

@misc{lacuna_local_prediction_stability_4c28110c_v1,
  title = {local prediction stability},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/local-prediction-stability/rdr_4c28110c363f4b31a45be88ef0231f16/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-19},
}

See something wrong? Report a mistake