MindDial: Enhancing Conversational Agents with Theory-of-Mind for Common Ground Alignment and Negotiation

Shuwen Qiu, Mingdian Liu, Hengli Li, Song-Chun Zhu, Zilong ZhengSIGDIAL 2024 · OpenReview

TL;DR

This paper introduces MindDial, a conversational AI framework that equips large language models (LLMs) with "Theory of Mind" (ToM) to better align and negotiate common ground in dialogues. By explicitly modeling speaker beliefs and their predictions of the listener's beliefs, MindDial significantly improves task success in cooperative and negotiation scenarios, making AI interactions more socially realistic.

What Problem Does This Paper Address?

Humans naturally build and maintain "common ground" – shared understanding, beliefs, and intentions – during conversations. This is crucial for effective communication, allowing us to be more concise and efficient. While modern LLMs excel at generating fluent text, they often struggle with this nuanced aspect of human interaction. They tend to operate on a single turn of understanding, failing to account for individual differences in context, prior knowledge, or the other participant's mental state. This limitation hinders their ability to truly align on topics or negotiate effectively, especially in tasks requiring shared goals or resolving differing perspectives. The paper aims to bridge this gap by imbuing conversational agents with a form of "Theory of Mind" (ToM) – the ability to attribute mental states, beliefs, and intentions to others.

Methods

The core innovation of MindDial is its explicit "mind module" designed to estimate beliefs. This module tracks three levels of belief:

  1. First-order belief: The agent's own understanding of the situation or goal (e.g., who the mutual friend is, or how to split items).
  2. Second-order belief: The agent's prediction of what the other agent believes about the situation or goal.
  3. Belief gap: The difference between the first and second-order beliefs, highlighting areas of misalignment or misunderstanding.

This mind module is integrated into a two-step response generation process. First, the mind module estimates these beliefs based on dialogue history and private knowledge. Then, a response generator (either a fine-tuned LLM or a prompted LLM) uses these belief estimations to craft a response aimed at resolving any identified belief differences and advancing the task goal. The framework is flexible and can be applied to existing LLMs.

Datasets

The paper evaluates MindDial on two established dialogue datasets:

  • MutualFriend: A cooperative alignment task where agents try to identify a shared mutual friend based on private friend lists.
  • CaSiNo: A negotiation task where agents must distribute camping supplies to maximize their individual gains, reflecting a semi-cooperative scenario.

Key Insights and Ideas

  • Theory of Mind is Crucial for Dialogue: The research strongly suggests that even basic forms of ToM (modeling others' beliefs) are essential for improving performance in complex dialogue tasks like alignment and negotiation, going beyond simple language generation.
  • Beyond Single-Turn Understanding: LLMs' current limitations in dialogue stem from their inability to maintain a persistent model of the other agent's mental state across turns. MindDial directly addresses this by explicitly tracking beliefs.
  • Three-Level Beliefs Matter: The study demonstrates that considering both the agent's own beliefs and their prediction of the other agent's beliefs (and the gap between them) is more effective than using only one level of belief estimation.
  • Flexibility in Application: MindDial can be integrated with both fine-tuning and prompting-based LLM approaches, showcasing its adaptability.

Key Results

  • Improved Task Success: Models equipped with MindDial's mind reasoning capabilities significantly improved success rates in both MutualFriend (alignment) and CaSiNo (negotiation) tasks across various LLMs (LLaMA, GPT-3.5, GPT-4). For instance, in MutualFriend, success rates increased substantially compared to models without mind reasoning.
  • Enhanced Efficiency: In alignment tasks, mind-enabled models achieved higher success rates per turn, indicating more efficient conversations.
  • Better Negotiation Outcomes: In CaSiNo, MindDial led to higher individual and collective negotiation scores, increased agreement rates, and a greater likelihood of achieving Pareto Optimal deals.
  • Belief Accuracy Correlates with Performance: The study found a strong correlation between the accuracy of the mind module's belief predictions and the overall task success rate.
  • Human Evaluation: Human subjects rated agents using MindDial more favorably in terms of cooperativeness, negotiation skills, and overall enjoyment compared to agents without mind reasoning.

Limitations and Future Work

  • Prompting Design Dependency: The effectiveness of the prompting-based approach relies on a well-defined knowledge base and goal. This might limit its generalization to more casual, open-ended conversations.
  • Belief Estimation Precision: The overall task success is highly dependent on the accuracy of the belief estimation. Future work should focus on developing more robust and accurate mind modules.
  • Generalization to Casual Dialogue: Extending MindDial to handle more informal and less structured conversational scenarios remains an open challenge.
  • Deeper ToM Inferences: While the models show promise in predicting beliefs, advancing to higher-level ToM inferences (e.g., understanding deception or complex social dynamics) is a key area for future research.

Summary updated

Abstract

Humans talk in daily conversations while aligning and negotiating the expressed meanings or common ground. Despite the impressive conversational abilities of the large generative language models, they do not consider the individual differences in contextual understanding in a shared situated environment. In this work, we propose MindDial, a novel conversational framework that can generate situated free-form responses to align and negotiate common ground. We design an explicit mind module that can track three-level beliefs – the speaker’s belief, the speaker’s prediction of the listener’s belief, and the belief gap between the first two. Then the next response is generated to resolve the belief difference and take task-related action. Our framework is applied to both prompting and fine-tuning-based models, and is evaluated across scenarios involving both common ground alignment and negotiation. Experiments show that models with mind modeling can generate more human-like responses when aligning and negotiating common ground. The ablation study further validates the three-level belief design can aggregate information and improve task outcomes in both cooperative and negotiating settings.
BibTeX
@inproceedings{qiu2024minddial,
  author = {Shuwen Qiu and Mingdian Liu and Hengli Li and Song-Chun Zhu and Zilong Zheng},
  title = {{MindDial: Enhancing Conversational Agents with Theory-of-Mind for Common Ground Alignment and Negotiation}},
  booktitle = {SIGDIAL},
  year = {2024},
  url = {https://openreview.net/forum?id=JotzvC3itk},
}
Figure