In autonomous systems, "knowing what you don't know" is the first step toward safety and efficiency. Traditionally, researchers have used uncertainty metrics—such as the variance of a model's predictions—to trigger interventions, such as asking a human for help or pausing a robot's execution. However, a static threshold (e.g., "ask for help if variance > 0.5") is often brittle. As a model learns, its baseline uncertainty changes, and a threshold that was appropriate during early training may become far too conservative later on.
Recent research has converged on adaptive uncertainty thresholding, a family of techniques that dynamically adjust the boundaries of trust based on the model’s evolving performance and the statistical distribution of its recent experiences.
The Problem with Static Boundaries
Static thresholds fail because they do not account for the relative nature of confidence. In reinforcement learning, for instance, Chen et al. et al. note that training from scratch is slow, but human experts are expensive. If an agent uses a fixed threshold to query an expert, it might waste queries on states that are difficult but manageable, or fail to query in new environments where its overall confidence has shifted.
Similarly, in robotic object learning, Villamizar et al. et al. argue that a static confidence boundary leads to user fatigue. As a robot becomes more proficient, it should naturally become more autonomous. A static threshold prevents this transition, forcing the human to remain "in the loop" even when the robot is capable of learning on its own.
Quantile-Based Trust
One of the most robust ways to adapt a threshold is to ground it in the model's recent history using quantiles or percentiles. Rather than picking an arbitrary number, the system looks at the distribution of uncertainty values it has seen lately and sets the threshold at a specific percentile.
Frauenknecht et al. et al. demonstrate this in Trust the Model Where It Trusts Itself - Model-Based Actor-Critic with Uncertainty-Aware Rollout Adaption. Their algorithm, MACURA, determines how long a simulated rollout should last by calculating a quantile of uncertainty values observed at the start of the rollout. By anchoring the threshold to the model's performance on known real-world data, the system automatically extends its "imagination" as the world model improves.
A similar logic is applied by Park et al. et al. in the context of vision-language-action models. Their framework, $T^3VF$, uses an adaptive buffer to maintain a history of recent uncertainty. By triggering updates only when current uncertainty falls within a low percentile of this distribution, the model ensures it only learns from reliable, high-confidence data during test-time adaptation.
Performance-Driven Adaptation
Another approach is to tie the threshold directly to the system's success rate. Villamizar et al. et al. propose a mechanism in Modeling robot's world with minimal effort that shrinks the uncertainty window based on the ratio of correctly classified samples to total requested samples. This creates a feedback loop: as the classifier's performance improves, the threshold for human intervention decreases, progressively reducing the burden on the human supervisor.
In dynamic environments, this adaptation can also work in reverse. Serrao et al. et al. show that for sentiment prediction in data streams, the threshold should be lowered when the learner is less confident due to data drift. This allows the system to query more instances when the environment changes, then raise the threshold again once it has stabilized.
Uncertainty-Driven Update Scheduling
Adaptive thresholds do more than just trigger human intervention; they can control the internal learning dynamics of the model itself. Yang et al. et al. explore this in Exploring Sparse Visual Prompt for Domain Adaptive Dense Prediction. By modulating the rate of parameter updates based on aggregate input-level uncertainty, their system balances rapid adaptation to new environments (like rain or night) with the preservation of stable knowledge.
This concept of "scheduling" is also central to the work of Sun et al. et al., who use a self-confidence metric—combining Monte Carlo Dropout uncertainty with data entropy—to decide when a model has finished learning one data distribution and is ready to advance to the next.
Bridging the Gap to the Real World
Adaptive thresholding is particularly effective for "sim-to-real" transitions. Mei et al. et al. introduced UGAT, which uses a dynamic threshold based on a moving average of past uncertainties to decide when to trust transformed actions in traffic signal control. This prevents the system from over-relying on simulated logic when the real-world dynamics are too divergent.
Finally, Singi et al. et al. highlight that these thresholds can be adjusted post-training. In their HULA framework, the tradeoff between autonomy and task success can be modulated at test time by shifting the uncertainty threshold, allowing practitioners to meet specific operational constraints without retraining the underlying model.
By moving from static to adaptive boundaries, these researchers have created systems that are not just aware of their own uncertainty, but are capable of contextualizing that uncertainty within their own learning progress and environmental changes.
Key Foundational Work
Interactive multiple object learning with scanty human supervision (2016) — contributed 2 concept elements to this direction.
Modeling robot's world with minimal effort (2015) — contributed 2 concept elements to this direction.