Abstract
Large language models (LLMs) showcase many desirable traits for intelligent and helpful robots. However, they are also known to hallucinate predictions. This issue is exacerbated in robotics where LLM hallucinations may result in robots confidently executing plans that are contrary to user goals, relying more frequently on human assistance, or preventing the robot from asking for help at all. In this work, we present LBAP (Likelihood-Based Alignment for Planners), a novel approach for utilizing off-the-shelf LLMs alongside a novel Action feasibility metric (A-Feasibility) in robotic planners that minimize harmful hallucinations and human intervention. A-Feasibility measures whether a given action is possible and safe in the provided scene, helping to mitigate hallucinations and better align the LLM's confidence with the probability of success. Through experiments in simulation and the real world, we show that LBAP significantly increases success rate and decreases the amount of human intervention required relative to prior art.
Introduction
While Large Language Models (LLMs) like GPT-4 can generate complex plans from natural language instructions, they suffer from a lack of "grounding." An LLM might confidently suggest "pick up the coffee cup" even if no cup exists in the room, or "put the metal bowl in the microwave," which is a safety hazard. In robotics, this leads to a failure in uncertainty alignment: ideally, a robot should execute a task if it’s certain and ask for help if it’s confused. Current state-of-the-art methods rely on the LLM's internal "confidence" (next-token probabilities), but these values are often poorly calibrated for physical environments.
The Challenge of LLM Hallucinations
The core problem is that LLMs are often "confidently wrong." An LLM might give a hallucinated action a high probability simply because it sounds linguistically plausible in the context of the prompt. This prevents the robot from "knowing what it doesn't know," leading to either dangerous autonomous actions or unnecessary interruptions for human help.
LBAP: Grounding LLMs through Bayesian Inference
The researchers introduce LBAP, which treats the planning problem as a Bayesian update. Instead of just trusting the LLM's initial guess, they refine it using three components that define the A-Feasibility of an action:
- The Prior ($P(y_i)$): The raw probability assigned by the LLM to a candidate action in a Multiple Choice Question Answering (MCQA) format.
- Scene Grounding ($P(S|y_i)$): A check to see if the objects required for the action actually exist in the environment, using textual descriptions or vision models like ViLD or CLIP.
- World Knowledge ($P(W|y_i)$): A separate LLM prompt that evaluates if an action is safe and possible (e.g., "Is it safe to put this in the microwave?").
By multiplying these terms ($P(y_i|S,W) \propto P(y_i)P(S|y_i)P(W|y_i)$), the robot generates a posterior probability. If the resulting "prediction set" of likely actions contains more than one option (or zero), the robot asks the human for help. This structured approach makes the system more explainable, as users can see exactly why a robot rejected an action (e.g., "Object not found" vs. "Unsafe action").
Experimental Evaluation
LBAP was evaluated using two primary benchmarks:
- Simulation (Tabletop Rearrangement): A PyBullet environment where a robot arm moves blocks and bowls, featuring injected ambiguities regarding attributes (color/shape), numbers, and spatial relations.
- Real-World (Mobile Manipulator): A ClearPath TurtleBot operating in a kitchen environment, handling complex scenarios involving recycling, composting, and appliance safety.
Key Results
- Simulation: LBAP reduced the human help rate by 70% (from 30.1% to 8.9%) at a 60% success rate compared to the KnowNo baseline.
- Real-World: In kitchen tasks, LBAP achieved a 70% success rate while requiring help only 42% of the time, compared to 63% for the baseline—a 33% improvement in autonomy.
- Scaling Benefits: Interestingly, the gap between LBAP and prior methods widened with more powerful models like GPT-4. This suggests that as LLMs get smarter, the need for structured Bayesian grounding becomes even more critical to harness that intelligence safely.
Conclusion
LBAP demonstrates that separating planning from feasibility and safety checks allows robots to leverage the intelligence of LLMs while remaining grounded in physical reality. By aligning robot confidence with actual environmental constraints, LBAP significantly reduces the need for human intervention. Future work will explore more complex, long-horizon tasks and tighter integration with real-time, dynamic perception systems.
Summary updated