A Plan-Based Model for Response Generation in Collaborative Task-Oriented Dialogues

Jennifer Chu-Carroll, Sandra CarberryAAAI 1994

This 1994 classic from the AAAI proceedings addresses a fundamental challenge in Human-Computer Interaction: how should an AI behave when it recognizes that a user is making a mistake? Rather than acting as a passive respondent, the authors propose a model for a truly collaborative "consultant" capable of negotiating, disagreeing, and suggesting superior alternatives.

The Challenge of Collaborative Dialogue

In the early 1990s, most "cooperative" dialogue systems functioned like sophisticated vending machines: a user asked a question, and the system provided a direct answer. However, human experts do not operate this way. If a student asks, "Who is teaching AI?" because they believe it satisfies a seminar requirement, a human advisor would not simply name the professor; they would point out that AI does not actually fulfill that specific requirement.

To be a true collaborator, a system must be able to:

  1. Detect Infeasibility: Identify when a user's proposed action is "ill-formed" or impossible.
  2. Recognize Sub-optimality: Determine when a user's plan is valid but fails to align with their own stated preferences.
  3. Negotiate: Address these discrepancies without breaking the flow of the conversation or confusing the dialogue state.

A Multi-Level Architecture

The researchers developed a plan-based architecture centered on a component called the Arbitrator. The system interprets every user utterance as a proposal to add actions to a shared plan. To manage this, the system tracks the conversation across four distinct levels:

  • Domain Level: The concrete task at hand (e.g., taking a specific course).
  • Problem-Solving Level: The process of building the plan (e.g., "let's find a course that fits your schedule").
  • Discourse Level: The communicative acts themselves (e.g., "Informing" or "Asking").
  • Belief Level: An innovation that tracks the underlying beliefs proposed by utterances. This allows the system to disagree with the reasoning behind a statement without losing track of the conversation's progress.
The multi-level dialogue model mapping user utterances to domain and problem-solving goals.

The Propose-Evaluate-Modify Cycle

When a user makes a proposal, the system enters a cycle managed by the Arbitrator. First, the Evaluator checks the proposal for two criteria:

1. Validity

Is the action possible? Does it actually lead to the user's stated goal? If a student proposes taking a course that is already full or doesn't meet a requirement, the system flags it as invalid.

2. Optimality

Even if a plan is valid, is there a better way? The system employs a Ranking Advisor that uses an additive model of human decision-making. It weighs user preferences—such as a desire for easy courses, specific time slots, or interesting content—against domain facts to score different options.

The Ranking Advisor's additive model for evaluating course optimality based on user preferences.

If the Evaluator finds a flaw, the Modifier takes over. Rather than simply overwriting the user's plan, it initiates a "meta-plan" to resolve the conflict via natural language, effectively "squaring away" beliefs before moving forward.

Key Insights

  • Disagreement as a Responsibility: The paper argues that a system that fails to point out a user's mistake is failing its responsibilities as a participant in collaborative problem-solving.
  • Recursive Negotiation: The act of modifying a proposal is itself a proposal. This allows for recursive loops where the system proposes a fix, the user evaluates that fix, and they negotiate the correction itself.
  • The Necessity of the Belief Level: By separating the belief (e.g., "Dr. Brown is a full professor") from the discourse act (e.g., "The user said Dr. Brown is a full professor"), the system avoids misattributing logic to the user when attempting to correct underlying misconceptions.

Conclusion and Legacy

The authors successfully demonstrated a system that could correct invalid plans, suggest better alternatives based on weighted preferences, and handle multi-level conflicts (e.g., "That course is full" vs. "Being a full professor doesn't mean he teaches AI"). While the symbolic, plan-based nature of the system requires a highly structured "recipe library" that limits its scalability to open-domain conversation, its conceptual framework for collaborative negotiation remains a touchstone for modern task-oriented dialogue research.

Summary updated

BibTeX
@inproceedings{chucarroll1994plan,
  author = {Jennifer Chu-Carroll and Sandra Carberry},
  title = {{A Plan-Based Model for Response Generation in Collaborative Task-Oriented Dialogues}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {1994},
}
Figure