Collective Intelligence in Human-AI Teams: A Bayesian Theory of Mind Approach

Samuel Westby, Christoph RiedlAAAI 2023 · DOI

TL;DR

This paper introduces a Bayesian "Theory of Mind" (ToM) agent that can infer teammates' mental states from communication in human-AI teams. This agent not only predicts human behavior but also enables AI interventions that significantly boost team performance, offering a novel, real-time measure of social perceptiveness.

What Problem Does This Paper Address?

The increasing integration of AI into collaborative teams promises to enhance human capabilities. However, developing AI agents that can effectively interact with multiple, diverse human teammates in complex cognitive tasks remains a significant challenge. A key missing piece is the AI's ability to understand and model the mental states (beliefs, opinions, knowledge) of its human collaborators, a skill humans possess through "Theory of Mind" (ToM). Existing methods for measuring ToM are often static and indirect, limiting their application in dynamic team settings. This paper aims to bridge this gap by developing an AI agent with a computational ToM and a dynamic measure of this ability to improve human-AI team performance.

Methods

The core of this work is a networked Bayesian agent that models teammates' mental states. Each agent is paired with a human teammate and observes their communication. The agent maintains two types of mental models:

  • Ego Model: Represents the agent's own beliefs about the task.
  • Alter Models: Represents the agent's beliefs about each of its teammates' beliefs.

These models are updated using Bayesian inference, incorporating observations from both the initial task clues and incoming messages. A crucial parameter, $\alpha_D$ (ToM ability), modulates how the agent weighs its own beliefs (Ego Model) against its inferred beliefs of others (Alter Models). This parameter is learned from data and is central to the agent's ability to infer and predict human behavior.

The authors also introduce a novel, real-time measure of communication ToM ($\alpha_C$). This measure quantifies an individual's ability to send messages that are informative and relevant to their teammates' inferred mental states.

Finally, they propose AI-triggered interventions. The agent identifies divergences between its Ego Model and Alter Models, signaling potential misunderstandings or missed information. It then selects and sends messages to neighbors that minimize the "KL divergence" (a measure of difference between probability distributions) between its own and its teammates' inferred beliefs, aiming to improve collective understanding.

Datasets

The study utilizes data from an IRB-approved online experiment involving 145 individuals organized into 29 human-only teams of five. These teams participated in a Hidden Profile task, a cognitive decision-making scenario where information is distributed among teammates, requiring effective communication to solve. Teams were assigned to various communication topologies (network structures). The data consists of text-based chat messages exchanged during the task.

Key Insights and Ideas

  • Humans struggle with information integration: Even with clear communication channels, humans tend to underweight ambiguous information and struggle to fully integrate diverse perspectives, leading to suboptimal team performance.
  • ToM is a critical driver of collective intelligence: The ability to infer teammates' mental states is a strong predictor of both individual and team performance.
  • Dynamic ToM measurement is possible: The proposed $\alpha_C$ measure, derived from communication patterns, provides a real-time, group-level assessment of social perceptiveness, outperforming traditional static measures.
  • AI can effectively augment human teams: By modeling human ToM and intervening strategically, AI agents can significantly improve team outcomes, even in ad-hoc teams with unfamiliar members.
  • Cognitive biases are quantifiable: The model's ability to capture human decision biases (e.g., under-weighting ambiguous information) offers a pathway to understanding and mitigating them.

Key Results

  • The Bayesian ToM agent's model fits human behavior significantly better than simpler models, demonstrating the importance of both self- and partner-actualization loops.
  • The agent's predictions of human decisions are highly accurate, even capturing errors due to task difficulty and network position.
  • The real-time communication ToM measure ($\alpha_C$) explains 8% of the variation in final team performance after observing only the first 25% of messages, a 170% improvement over the current state-of-the-art (e.g., the "Reading the Mind in the Eyes" test, which explains ~3%).
  • AI-triggered interventions led to a 4.9% performance increase in human-AI teams compared to human-only teams, reaching 82.1% accuracy.
  • High-performing teams exhibit distinct communication patterns, sending high-information-content messages early and consolidating later, while low-performing teams struggle with this dynamic.

Limitations and Future Work

  • Limitations: The study acknowledges that humans do not perfectly model each other's mental states and exhibit cognitive biases. The current intervention strategy is based on minimizing KL divergence, which could be further refined. The analysis of the Hidden Profile task focused primarily on the "culprit" dimension for simplicity.
  • Future Work: The authors suggest extending the framework to incorporate a fuller implementation of the free energy principle for learning asymmetric beliefs about partner reliability. They also propose exploring how AI can learn to modulate shared cognition dynamically, similar to high-performing human teams, and further investigate the emergence of collective memory, attention, and reasoning. The potential for AI to augment human cognitive limitations in ad-hoc teams without pretraining is highlighted as a key future direction.

Summary updated

BibTeX
@inproceedings{westby2023collective,
  author = {Samuel Westby and Christoph Riedl},
  title = {{Collective Intelligence in Human-AI Teams: A Bayesian Theory of Mind Approach}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2023},
  doi = {10.1609/aaai.v37i5.25755},
}
Figure