When humans give instructions to robots, they rarely speak with mathematical precision. Instead, human communication relies heavily on shared context, omitting steps deemed obvious or using abstract descriptors like "walk carefully." This phenomenon—known as linguistic underspecification or instructional ambiguity—creates a fundamental challenge for autonomous agents. A robot cannot execute a statistical guess without risking failure or dangerous behavior.
To bridge this gap, a prominent research direction focuses on treating natural language not as a direct, deterministic command, but as a noisy, underspecified observation of human intent. By explicitly modeling, quantifying, and resolving this ambiguity, modern frameworks enable robots to operate safely and reliably in complex, real-world environments.
The Challenge of Linguistic Underspecification
The core difficulty in human-robot interaction stems from the "semantic gap" between qualitative human language and the quantitative requirements of robotic control systems. As Nobuyuki Shimizu notes in the context of navigation, human route instructions frequently utilize unrestricted vocabulary and omit intermediate steps. Daqing Yi demonstrates that even when mapping qualitative commands (like "walk carefully") to path-planning objectives, multiple distinct physical trajectories can satisfy the same human request, leaving the robot's trajectory planner under-constrained.
Furthermore, human communication is inherently lossy. Anjali Narayan-Chen et al. highlight that these omitted details prevent agents from reliably bridging the gap between text and grounded physical actions. This problem compounds in cluttered environments, leading to referential ambiguity. For instance, if a user says "pick up the shirt" in a room containing multiple similar garments, the linguistic query lacks the granularity required for unique target identification, causing what Zhenyang Li et al. term instance confusion.
Quantifying What the Robot Doesn't Know
To resolve ambiguity, an autonomous agent must first recognize that an instruction is unclear. Rather than failing silently or executing a dangerous action based on a statistical guess, modern frameworks equip robots with mechanisms to evaluate their own uncertainty.
One foundational approach relies on information theory. Stefanie Tellex et al. model the problem by calculating the entropy of "grounding" variables within a probabilistic graphical model. By computing this entropy across the environment, the robot can mathematically pinpoint exactly which phrase or object reference is the source of the greatest uncertainty.
[Human Instruction] ---> [Probabilistic Graphical Model] ---> [High Entropy Detected] ---> [Targeted Clarification]
(Uncertain Grounding)
With the rise of Large Language Models (LLMs) in robotics, uncertainty estimation has shifted toward probabilistic sampling. Jeongeun Park 0002 et al. introduce a context sampling technique to measure LLM uncertainty, allowing the agent to classify whether a user command is certain, vague, or physically impossible. Similarly, Thierry Deruyttere et al. emphasize that detecting these underspecified commands is a prerequisite for safe deployment, ensuring autonomous vehicles or manipulators seek clarification rather than acting on ambiguous directives.
Active Disambiguation via Targeted Dialogue
Once an agent identifies an ambiguity, the most direct way to resolve it is to ask the human for more information. However, generic questioning (e.g., "What should I do next?") is highly inefficient. The goal is to minimize the length and complexity of the clarifying dialogue.
Brojeshwar Bhowmick et al. address this by comparing structured semantic graphs of the user's instructions against grounded instances in the environment. This structural comparison allows the robot to diagnose the exact nature of the ambiguity—such as identifying whether there are missing parameters or multiple matching target objects—and generate highly precise, human-intelligible clarification queries.
By transforming agents from passive instruction-followers into active conversational participants, frameworks like the one proposed by Qiaozi Gao et al. demonstrate significant leaps in task success rates. When an agent can proactively ask "Where is the knife?" or "Which plate?", it eliminates the need for the human to provide perfectly specified initial instructions.
Breaking Ties with Multimodal Context
Dialogue is not always the most efficient channel for resolving ambiguity; nonverbal cues can provide immediate spatial grounding where words fail. When multiple identical objects satisfy a verbal description, physical context and human behavior can act as natural tie-breakers.
Boyang Wang demonstrates how pairing deictic commands (such as "put this in that") with simple 2D pointing gestures or clicks can instantly resolve environmental ambiguities that stymie text-only models. This multimodal approach bypasses the need for complex, verbose verbal descriptions of an object's precise location or appearance.
Beyond explicit gestures, implicit human behaviors like gaze can be leveraged. Chiara Falagario et al. integrate human gaze estimation directly into a robotic vision architecture. When a user issues an ambiguous command like "pass me the box" in a scene containing multiple boxes, the system uses the user's visual focus as a real-time spatial constraint to isolate the intended target.
Mental Simulation and Latent Specifications
A final strategy for managing ambiguity involves internal reasoning and simulation before physical execution. If a robot can project the future consequences of an instruction, it can evaluate whether the command provides a clear, executable path.
Chris Paxton et al. introduce this concept as "prospection"—a process where an agent mentally rehearses a sequence of interpretable subgoals and "hallucinates" the resulting visual world states. By predicting the future states required to fulfill a command, the system can determine if a high-level human intent maps cleanly to low-level robotic control parameters, catching ambiguities before the physical hardware ever moves.
[Qualitative Command] ---> [Mental Rehearsal / Prospection] ---> [Predict Visual Subgoals] ---> [Execute Trajectory]
When language is too abstract to capture spatial nuances or highly specific user preferences, researchers have looked toward alternative intermediate representations. Huaxiaoyue Wang et al. address this by recursively summarizing long, unconstrained human demonstrations into concise "latent specifications," which are then expanded into hierarchical code. This approach bypasses the limitations of purely text-based instructions, allowing the robot to learn personalized, long-horizon tasks that language alone cannot fully describe.