Abstract
Large-scale robotic "foundation models" trained on diverse datasets have shown significant promise in generalization across tasks and platforms. However, these models often exhibit overconfidence, leading to brittle performance and failures when deployed in novel environments or in the presence of distractors. This paper introduces a simple, training-free deployment strategy designed to mitigate these issues. By combining temperature scaling for post-hoc calibration with a neighborhood-based action selection mechanism, the proposed method improves success rates and robustness across multiple state-of-the-art imitation learning architectures without requiring expensive model re-training.
Introduction
The emergence of language-conditioned imitation learning policies, such as Perceiver-Actor (PerAct), RVT, and CLIPort, has enabled robots to perform complex manipulation tasks from natural language instructions. Despite their capabilities, these models frequently lack well-calibrated uncertainty. In practice, a model may assign a high probability to an incorrect action, such as grasping an object at an unstable edge or being distracted by irrelevant visual noise. This lack of reliability poses a significant barrier to real-world deployment. This research addresses the challenge by making pre-trained models "aware" of their own uncertainty during the inference phase.
Methodology
The authors propose a two-step, post-hoc modification to the standard deployment pipeline that requires no changes to the underlying model weights.
1. Target Task Calibration
To address miscalibration, the researchers employ temperature scaling. This technique involves finding a scaling factor ($T$) that adjusts the model's output logits. By utilizing a small set of expert demonstrations (as few as 25) from the specific target task, the model's predicted confidence is aligned with the actual likelihood of action success. This ensures that the probability scores generated by the model are representative of its true performance accuracy.
2. Uncertainty-Aware Action Selection
Standard deployment typically follows a "greedy" approach, selecting the single action with the highest confidence score. This paper proposes a Neighborhood-based Action Selection method. Instead of relying on an isolated peak, the algorithm calculates a new score for candidate actions by summing the confidence scores of all similar actions within a defined radius ($ au$).
This approach favors regions of the action space where the model is consistently confident. By aggregating local confidence, the policy becomes more conservative and precise—for instance, preferring the center of an object where many neighboring points are also valid, rather than an edge where a slight deviation would lead to failure.
Experimental Evaluation
The methodology was evaluated across three state-of-the-art models and two primary benchmarks:
- RLBench: Used to test Perceiver-Actor (PerAct) and RVT models across 18 manipulation tasks, including "open drawer" and "push buttons."
- Ravens Benchmark: Used to evaluate CLIPort on tasks such as block stacking and kit assembly.
- Distractor Robustness: The authors introduced modified Ravens tasks containing irrelevant objects to evaluate the models' ability to ignore out-of-distribution noise.
Key Insights
Calibration and Model Architecture
The study revealed that calibration needs vary by model. CLIPort, which was trained on 1,000 demonstrations per task, was found to be relatively well-calibrated out of the box. In contrast, PerAct and RVT, trained on 100 demonstrations per task, exhibited significant overconfidence. For these models, temperature scaling was essential, drastically reducing the Expected Calibration Error (ECE). For example, in the "Light Bulb" task, PerAct's ECE dropped from 0.64 to 0.07.
Robustness to Distractors
Visual distractors often create isolated "spikes" of high confidence in a model's internal heatmap. Because these spikes lack a supportive neighborhood of high-confidence actions, the proposed neighborhood-summing algorithm effectively filters them out. In the "stack-block" task, the baseline CLIPort model's performance dropped from ~95% to 79.6% when distractors were present. With uncertainty-aware selection, performance was restored to 97.1%.
Results
The proposed deployment strategy yielded consistent improvements:
- RLBench Success Rates: PerAct saw an increase from 38.2% to 41.4%, while RVT improved from 60.2% to 62.3%.
- Precision: The neighborhood aggregation naturally encouraged more robust grasping and placement by favoring central, high-density confidence regions.
Conclusion and Future Work
This work demonstrates that the reliability of robotic foundation models can be significantly enhanced through simple post-hoc calibration and smarter action selection. While the method cannot correct a model that has failed to learn the task entirely, it provides a vital layer of robustness for deployment. Future research may explore learning the similarity metrics ($d$) and thresholds ($ au$) directly from data, or extending these uncertainty-quantification techniques to the domain of dynamics forecasting.
Summary updated