Explicit World Models for Reliable Human-Robot Collaboration

Kenneth Kwok, Basura Fernando, Qianli Xu, Vigneshwaran Subbaraju, Dongkyu Choi, Boon Kiat Quek

Abstract

This paper addresses the challenge of achieving robustness in embodied AI under conditions of sensing noise, ambiguous instructions, and complex human-robot interaction. We propose a departure from traditional formal verification methods, which often prioritize model predictability in isolation. Instead, we emphasize the dynamic, subjective nature of human-robot collaboration (HRC), arguing that reliability is contextually determined and must be aligned with human goals and expectations. We advocate for a fundamentally different approach centered on building and updating an accessible "explicit world model." This model represents the common ground between human and AI, ensuring that robot behaviors remain consistent, comprehensible, and socially aligned.

Introduction

As robots transition from controlled industrial settings to fluid human environments, the definition of "reliability" must evolve. Current trends in embodied AI favor end-to-end deep learning models—opaque black boxes that often fail unpredictably when faced with the nuances of human social behavior. While researchers frequently turn to formal verification to solve these issues, such methods struggle to account for the inherent ambiguity of human intentions.

This position paper argues that achieving reliable HRC requires robots to maintain explicit world models: interpretable, neuro-symbolic representations built in real-time. By establishing a dynamic "common ground" through multimodal cues like gaze, gestures, and language, robots can move beyond statistical next-token prediction toward true collaborative intelligence.

The Problem: The Limits of Black-Box Reliability

In human-centric spaces, a robot faces a constant stream of noisy sensor data and vague instructions. A command like "pass me that" is meaningless without context provided by the speaker's gaze or a pointing gesture. Purely neural, end-to-end architectures place the entire burden of reliability on the model's ability to learn every possible permutation of these interactions—a task that is nearly impossible in open-ended environments.

Furthermore, reliability in a social context is relational. It is not an absolute metric but a measure of how well the robot’s actions align with the human partner's expectations. When a robot's internal logic is hidden, any deviation from expected behavior is perceived as a failure of reliability, even if the robot is technically following its training data.

Proposed Framework: Explicit World Modeling

To bridge this gap, we advocate for a neuro-symbolic approach that synthesizes the generative capabilities of neural networks with the logical rigor of symbolic reasoning. This framework rests on three pillars:

1. Multimodal Interaction and Fusion

Reliability begins with accurate perception. Rather than treating vision and speech as isolated inputs, our approach continuously integrates human cues—such as prosody, eye gaze, and pointing gestures—to disambiguate instructions. By using adaptive systems like COSM2IC and Ges3ViG, the robot can prioritize different modalities based on the context, such as relying more on gestures when verbal instructions are muffled or vague.

2. Neuro-Symbolic Architectures

We propose architectures like NeSyC (Neuro-Symbolic Composition), which utilize a feedback loop between Large Language Models (LLMs) and symbolic solvers. In this setup, the LLM generates inductive inferences about a task, while a symbolic solver (using Answer Set Programming) deductively validates those inferences against known physical or logical constraints. This ensures that the robot's plan is not just statistically likely, but logically sound.

3. Knowledge Module Learning (KML)

To ground symbolic reasoning in the physical world, we utilize Knowledge Module Learning. This involves encoding procedural knowledge into knowledge graphs that link tasks, actions, tools, and objects. This allows the agent to generate interpretable, stepwise reasoning traces. If a robot fails a task, a human can inspect these traces to understand why the error occurred, making the system debuggable and transparent.

Key Insights for Future HRC

Our synthesis of recent research highlights several critical insights for the field:

  • Common Ground is Essential: Effective collaboration is impossible without a shared understanding of the environment. This common ground is built through continuous, often involuntary non-verbal cues.
  • Explainability Equals Reliability: A system that can explain its reasoning is inherently more reliable in a human context than a high-performing black box. Neuro-symbolic frameworks allow for the creation of "verifiable" steps that humans can trust.
  • Relational Metrics: We must move toward evaluating robots based on their ability to align with human goals rather than just task completion rates in isolation.

Evaluation and Benchmarks

Standard benchmarks like VQA (Visual Question Answering) often fail to capture the dynamic context of real-world HRC. We advocate for the use of more specialized benchmarks, such as:

  • TCQA (Task-oriented Collaborative Question Answering): Evaluates grounding methods specifically in collaborative settings.
  • PKR-QA: Focuses on procedural reasoning to test an agent's ability to follow complex, multi-step tasks.

Conclusion and Future Work

Transitioning to explicit world models presents significant technical challenges, primarily the need to balance model complexity with real-time performance. However, the shift away from opaque, end-to-end models is necessary for the development of truly reliable embodied AI. We call for a multidisciplinary effort to create systems that can autonomously refine their internal representations through both physical and social embodiment, ultimately leading to robots that act as true partners in human environments.

Summary updated

Abstract

This paper addresses the topic of robustness under sensing noise, ambiguous instructions, and human-robot interaction. We take a radically different tack to the issue of reliable embodied AI: instead of focusing on formal verification methods aimed at achieving model predictability and robustness, we emphasise the dynamic, ambiguous and subjective nature of human-robot interactions that requires embodied AI systems to perceive, interpret, and respond to human intentions in a manner that is consistent, comprehensible and aligned with human expectations. We argue that when embodied agents operate in human environments that are inherently social, multimodal, and fluid, reliability is contextually determined and only has meaning in relation to the goals and expectations of humans involved in the interaction. This calls for a fundamentally different approach to achieving reliable embodied AI that is centred on building and updating an accessible "explicit world model" representing the common ground between human and AI, that is used to align robot behaviours with human expectations.
BibTeX
@misc{kwok2026explicit,
  author = {Kenneth Kwok and Basura Fernando and Qianli Xu and Vigneshwaran Subbaraju and Dongkyu Choi and Boon Kiat Quek},
  title = {{Explicit World Models for Reliable Human-Robot Collaboration}},
  year = {2026},
  eprint = {2601.01705},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2601.01705},
}
Figure