Mitigating Physical Hallucinations in Embodied Foundation Models

This direction is split into other directions: physical safety grounding. Its last version stays readable and citable here.

Version 1 of 131 papers38 conceptsfrozen 2026-10-04

Large Language Models (LLMs) are exceptionally good at reasoning about the world in the abstract. If you ask an LLM how to "clean a spill," it can generate a logical sequence of steps: find a paper towel, walk to the spill, and wipe it up. However, when these models are used to control physical hardware—a field often called Embodied AI or Agent AI—a new and dangerous failure mode emerges: the physical hallucination.

In a digital chat, a hallucination is a factual error. In robotics, a physical hallucination is a command that violates the laws of physics or the safety limits of hardware. As Yi Wu notes in SELP, "mostly correct" is not good enough when a drone is navigating a hallway; a single probabilistic slip can lead to a catastrophic collision.

The Planning-to-Execution Gap

The core challenge lies in the "planning-to-execution" gap. LLMs operate on tokens, while robots operate on torques, voltages, and coordinates. To bridge this, researchers like Yue Cao have developed techniques to ground natural language into specific position and force set-points using Task Frame Formalism.

Despite this grounding, models often suggest numerical values that are physically impossible—such as commanding a robot arm to move at a velocity that would strip its gears. These errors occur because the generative engine lacks an inherent understanding of physical limits. As explored in PaLM-E by Danny Driess and colleagues, even massive multimodal models with 562 billion parameters can produce unpredictable textual outputs that translate into unsafe physical actions.

Formal Verification as a Safety Shield

To solve this, researchers are moving away from letting LLMs "drive" robots directly. Instead, they are treating the LLM as a high-level planner whose output must be verified by a deterministic "safety layer."

One of the most rigorous approaches involves Formal Verification. Neel P. Bhatt and the team at UT Austin developed a framework in Joint Verification and Refinement of Language Models for Safety-Constrained Planning that converts LLM-generated Python code into mathematical models called automata. By applying a "Compositional Verification Theorem," they can mathematically prove whether a robot's plan is safe before the first motor turns. If the math doesn't check out, the plan is rejected and refined.

Hybrid Control and Parameter Tuning

Another strategy is to use the LLM not as a controller, but as an "advisor" to existing, proven algorithms. In the context of autonomous driving, S P Sharan introduced LLM-ASSIST, where the LLM tunes the parameters of a rule-based planner during high-uncertainty scenarios. This hybrid approach ensures the vehicle stays within safe physical boundaries defined by classical control theory while benefiting from the LLM’s high-level reasoning.

Similarly, in industrial manufacturing, Shihao Li utilizes LLMs to design control systems for roll-to-roll manufacturing, but passes all LLM suggestions through a "safety filter" to prevent the proposal of unstable parameters that could damage expensive equipment.

Visualizing the Invisible: Human Verification

In complex industrial settings where mathematical proofs might be too rigid, researchers are using Mixed Reality (MR) as a verification layer. Cathy Mengying Fang and researchers from MIT and Universal Robots developed a system where an LLM generates spatial waypoints, which are then projected via Augmented Reality for a human to "preview." This allows a non-expert operator to visually catch a physical hallucination—like a robot arm swinging through a support beam—before the command is executed.

Toward Action Tokenization

The field is currently shifting toward a unified framework called Vision-Language-Action (VLA) models. As Fengshuo Bai explains in a survey on Action Tokenization, the goal is to treat physical movements as "action tokens"—the embodied counterparts to words.

By discretizing continuous physical space into tokens that the model can "speak," researchers hope to bake physical constraints directly into the model's vocabulary. However, as Qiuyuan Huang argues in the position paper on Agent AI, achieving "Holistic Intelligence" requires more than just better tokens; it requires a fundamental shift toward models that are trained on interactive, physical datasets where the cost of a hallucination is a literal, not figurative, crash.

Papers in this version

History & citationVersion 1 · 2026-10-04

Split into other directions · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_mitigating_physical_hallucinations_in_em_10d4c813,
  title = {Mitigating Physical Hallucinations in Embodied Foundation Models},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/mitigating-physical-hallucinations-in-embodied-foundation-models/rdr_10d4c813c85944e08ecec2a2db38f283},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-21},
}

This exact version (version 1, frozen text):

@misc{lacuna_mitigating_physical_hallucinations_in_em_10d4c813_v1,
  title = {Mitigating Physical Hallucinations in Embodied Foundation Models},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/mitigating-physical-hallucinations-in-embodied-foundation-models/rdr_10d4c813c85944e08ecec2a2db38f283/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-21},
}