Mutual Understanding in Human-Machine Teaming

Rohan R. PalejaAAAI 2022 · OpenReview

Abstract

Collaborative robots (i.e., "cobots") and machine learning-based virtual agents are increasingly entering the human workspace with the aim of increasing productivity, enhancing safety, and improving the quality of our lives. These agents will dynamically interact with a wide variety of people in dynamic and novel contexts, increasing the prevalence of human-machine teams in healthcare, manufacturing, and search-and-rescue. In this research, we enhance the mutual understanding within a human-machine team by enabling cobots to understand heterogeneous teammates via person-specific embeddings, identifying contexts in which xAI methods can help improve team mental model alignment, and enabling cobots to effectively communicate information that supports high-performance human-machine teaming.

Introduction

As collaborative robots and AI agents transition from isolated tools to active teammates, they must navigate the complexities of human environments—from manufacturing floors to search-and-rescue operations. However, effective human-machine teaming (HMT) is currently limited by three primary challenges: the inherent heterogeneity of human decision-making, the opaque nature of "black-box" AI policies, and the risk of human cognitive overload during interaction. This research outlines a comprehensive framework to bridge these gaps, fostering mutual understanding through personalized learning, interpretable architectures, and strategic communication.

Methodology

To address the limitations of current HMT systems, we introduce three distinct methodological contributions:

1. Modeling Human Heterogeneity

Humans do not make decisions uniformly; they rely on varied, personalized heuristics developed over years of experience. Standard AI models often struggle to anticipate these diverse strategies. We developed a Heterogeneous Apprenticeship Learning framework—a data-efficient learning-from-demonstration system that utilizes variational inference and backpropagated "person-specific embeddings." This allows the model to learn shared task structures while automatically adapting to a specific human's unique decision-making criteria.

2. Interpretable Policy Architectures

To build trust, humans must be able to anticipate a robot's actions. We developed Personalized Neural Trees (PNT), an architecture specifically designed for sparsity and transparency. By using counterfactual reasoning via pairwise comparisons, the PNT discretizes learned policies into human-readable Boolean decision trees, making the AI's logic accessible to its human teammates.

3. Strategic Communication (MAGIC)

Effective teams communicate efficiently rather than constantly. To prevent cognitive overload, we developed MAGIC (Multi-Agent Graph-attentIon Communication). This end-to-end protocol emulates high-performing human teams by learning policies that dictate exactly when and with whom the robot should communicate, rather than broadcasting information indiscriminately.

Experimental Evaluation

We evaluated these methods using synthetic and real-world scheduling problem domains, as well as a complex, multi-level house-building scenario within the Minecraft domain to test ad-hoc teaming and xAI utility.

Key Results

  • Imitation Accuracy: The heterogeneous apprenticeship learning framework significantly outperformed state-of-the-art baselines, achieving a +51% increase in imitation accuracy in synthetic domains and an +11% increase in real-world scheduling tasks.
  • Interpretability: User studies confirmed that PNT models were significantly more interpretable, easier for humans to simulate mentally, and quicker to validate than standard neural networks.
  • Teaming Performance: In Minecraft-based trials, xAI support significantly improved Situational Awareness ($p < 0.05$) and boosted the performance of novices ($p < 0.05$).

The xAI Paradox and Cognitive Load

A critical insight from our research is that Explainable AI is a double-edged sword. While xAI generally improves situational awareness, its impact on performance is heavily dependent on the user's expertise. Our studies revealed that while novices benefit from explanations, the addition of xAI support significantly degraded the performance of experts ($p < 0.05$). For experts, the cognitive cost of processing an explanation often outweighed the benefit, leading to information overload and reduced efficiency.

Conclusion and Future Work

Achieving mutual understanding in HMT requires AI that is both adaptive and transparent. By utilizing person-specific embeddings and interpretable tree-based policies, we can create agents that respect human diversity and cognitive limits.

Future research will focus on extending the PNT model to handle continuous action spaces for more complex robotic control. Additionally, we plan to conduct larger-scale human-subject studies to test these frameworks in broader multi-agent environments involving heterogeneous groups of both humans and robots.

Summary updated

Abstract

Collaborative robots (i.e., "cobots") and machine learning-based virtual agents are increasingly entering the human workspace with the aim of increasing productivity, enhancing safety, and improving the quality of our lives. These agents will dynamically interact with a wide variety of people in dynamic and novel contexts, increasing the prevalence of human-machine teams in healthcare, manufacturing, and search-and-rescue. In this research, we enhance the mutual understanding within a human-machine team by enabling cobots to understand heterogeneous teammates via person-specific embeddings, identifying contexts in which xAI methods can help improve team mental model alignment, and enabling cobots to effectively communicate information that supports high-performance human-machine teaming.
BibTeX
@inproceedings{paleja2022mutual,
  author = {Rohan R. Paleja},
  title = {{Mutual Understanding in Human-Machine Teaming}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2022},
  url = {https://openreview.net/forum?id=4BTm8_V12z},
}
Figure