Domain Engineering to Represent Human Behavior Using Multi-Agent Planning and Inductive Methodologies

Salena Torres AshtonAAAI 2024 · DOI

Introduction

Traditional automated planning models are designed for computational efficiency, often prioritizing the most "optimal" path to a goal. However, in collaborative environments where humans and AI agents must work together, these models frequently fail to account for the "messy" reality of human behavior. Human interaction is characterized by redundancy, social nuance, and seemingly inefficient protocols that are, in fact, critical for maintaining shared mental models. This research introduces a novel domain engineering methodology designed to bridge this "representation gap," specifically within the high-stakes context of Urban Search and Rescue (SAR).

The Problem: The Representation Gap

When AI models abstract away human behaviors that appear redundant—such as repeating a question for clarity or engaging in social coordination—they lose the pragmatic weight of those actions. If an AI does not understand the latent mental models or social protocols humans use, it cannot accurately predict or support human teams. In SAR scenarios, providing guidance that is technically optimal but socially or practically inappropriate introduces significant ethical risks and reduces the overall effectiveness of the human-agent team.

An Interdisciplinary Methodology

To capture human behavior more authentically, the author proposes a four-part methodology that integrates insights from social sciences and formal AI planning:

  1. Multi-Agent Social Protocols: Utilizing existing frameworks to represent the formal commitments and coordination mechanisms between agents.
  2. Psycholinguistics of Question-Asking: The research focuses on questions as "information-rich" events. Questions provide asymmetrical insights into a collaborator’s beliefs, intentions, and uncertainties.
  3. Procedural Grounded Theory (GT): This inductive coding schema was used to annotate SAR data. Unlike deductive methods that force data into pre-existing categories, GT allows patterns of team planning to emerge directly from observed behavior.
  4. Hierarchical Task Networks (HTN): Annotated events are translated into HTNs and implemented using the Hierarchical Domain Definition Language (HDDL). This allows the multi-layered nature of human tasks to be mapped into a format that AI planners can process and execute.

Data and Reliability

The study utilized a dataset of 12 Urban Search and Rescue videos. Three autonomous annotators spent over 13 hours labeling these videos using the Procedural Grounded Theory approach. The process achieved an unweighted Cohen’s Kappa score of 0.892, indicating exceptionally high agreement and a reproducible methodology for capturing complex human interactions.

Key Insights: Inefficiency as a Feature

The research reveals that human "inefficiencies" are often functional components of collaboration:

  • The Power of Pragmatics: Social rules are vital for team cohesion. For example, a simple "no" might be factually correct but socially disruptive; understanding these nuances is essential for effective domain engineering.
  • Questions as Windows into the Mind: Teams that frequently use questions to clarify ideas demonstrate higher levels of collaboration and success than those that rely solely on instructions.
  • Consistency Over Volume: Performance in SAR simulations correlated more strongly with a consistent rate of question-asking rather than the total volume of communication. Teams that communicated steadily performed better than those who communicated sporadically.

Results and Formalization

The resulting HTN structures successfully formalized "non-optimal" behaviors, such as repeating questions and deviating from initial plans. The study found that teams engaging in these coordinated behaviors (even when not strictly required by the task) rescued more victims and achieved higher scores. This suggests that by modeling these nuances, AI agents can better align with human partners in real-world environments.

Future Work

As this research represents a PhD project in progress, future phases will focus on:

  • Empirical Evaluation: Directly comparing these HTN models against traditional planning models using specific performance metrics.
  • Causal Extraction: Incorporating text extraction to identify causal and counterfactual language, further refining the "Theory of Mind" within the AI models.
  • Interdisciplinary Expansion: Continuing to bridge the gap between formal semantics, social history, and AI planning to create more robust human-agent architectures.

Summary updated

BibTeX
@inproceedings{ashton2024domain,
  author = {Salena Torres Ashton},
  title = {{Domain Engineering to Represent Human Behavior Using Multi-Agent Planning and Inductive Methodologies}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2024},
  doi = {10.1609/aaai.v38i21.30391},
}
Figure