Impression Monitoring in Evaluation-Oriented Dialog - The Role of the Listener's Assumed Expectations and Values in the Generation of Informative Statements

TL;DR

This paper introduces IMP, a dialog system that goes beyond simply answering questions. It actively monitors the user's impression of an object, strategically volunteering additional information to shape that impression based on assumed user expectations and the system's own bias. This work is crucial for developing more sophisticated and human-like conversational agents that can engage in nuanced evaluative dialogues.

What Problem Does This Paper Address?

When we ask for information about something we're evaluating (like a rental apartment or a job candidate), we don't just want direct answers. We also implicitly consider what the informant isn't saying and what their motivations might be. This paper tackles the challenge of building a dialog system that can mimic this human-like understanding of evaluative conversations. Traditional systems might just provide factual answers, but IMP aims to understand and influence the impression a user forms by considering the informant's potential bias, the user's assumed expectations, and the pragmatic implications of what is said (and unsaid).

Methods

IMP employs a sophisticated strategy for generating informative statements in evaluation-oriented dialogues. Its core lies in:

  • Modeling Evaluator Expectations and Bias: The system is initialized with an "evaluation form" that represents the user's potential standards and prior expectations for the object being evaluated. It also operates with its own "actual bias" (e.g., positive, negative, or objective) and an assumption about the "projected bias" the user might attribute to it.
  • Impression Monitoring: IMP doesn't just answer. It actively monitors how its responses might shift the user's impression of the object. This involves calculating the "expected value" (how favorable the impression is likely to be) and "uncertainty" (how sure the user is about their impression).
  • Strategic Comment Selection: Based on the user's question and the system's internal models, IMP selects additional comments that are not directly asked for but are anticipated to have a desirable impact on the user's impression. This selection considers the comment's potential to shift the expected value and reduce uncertainty, while also accounting for the "effort" involved in making the comment and its "question-relatedness."
  • Pragmatic Interpretation: A key innovation is IMP's ability to anticipate how its statements (or the absence of statements) will be pragmatically interpreted by the user, considering the system's assumed bias. If a statement, when interpreted literally, would contradict the system's projected bias, IMP might choose a different comment or even infer that its projected bias was incorrect.

Datasets

Not discussed in the paper. The system uses an "evaluation form" which is described as a "particular explicit variant of a type of scale which one frequently sees used in practical contexts which require systematic evaluations." This form is supplied to the system in advance of each dialog and is not a dataset in the traditional sense.

Key Insights and Ideas

  • The Power of the Unsaid: The paper highlights that what an informant doesn't say can be as informative as what they do say. IMP's ability to anticipate how silence will be interpreted is a significant step towards more human-like dialog.
  • Bias as a Strategic Tool: Instead of trying to be perfectly neutral, IMP leverages its own bias (and the user's perception of it) as a strategic element in shaping impressions. This allows for more nuanced and potentially persuasive communication.
  • Balancing Directness and Influence: IMP demonstrates a delicate balance between directly answering questions and proactively influencing the user's overall evaluation. This is crucial for building trust and providing truly helpful information.
  • Pragmatic Interpretation is Key: Understanding how users infer meaning beyond the literal words is essential for effective dialog systems. IMP's mechanism for anticipating pragmatic interpretations is a core contribution.

Key Results

The paper presents a detailed trace of IMP's interaction in a simulated dialog about a room for rent (Figure 1). The results showcase IMP's ability to:

  • Provide direct answers to questions.
  • Volunteer additional, relevant comments that are not explicitly requested, such as "AND BY THE WAY THE ROOM IS VERY LARGE" or "IN FACT UNFORTUNATELY THERE'S A LOT OF NOISE FROM THE STREET."
  • Justify its comment selection based on anticipated impression changes, expected value shifts, and uncertainty reduction. For instance, it explains why a comment about noise might be made even if the direct answer is "NO" to a related question, due to the potential for a significant impression shift.
  • Handle situations where silence might be misinterpreted as an attempt to hide negative information, leading it to volunteer a comment to maintain an appearance of objectivity.

Quantitative results are not the primary focus; the paper emphasizes the conceptual framework and the qualitative demonstration of IMP's strategic dialog generation.

Limitations and Future Work

  • Limited Scope: IMP is described as a "restricted prototype" designed to explore specific topics without the overhead of a full Natural Language (NL) system. Its capabilities are limited to evaluation-oriented dialogs with specific speech act types and dialog goals.
  • Static Evaluation Form: The system assumes a fixed "evaluation form" for the user's expectations and biases, which are not updated during the dialog. More dynamic modeling of user beliefs would be beneficial.
  • Ad Hoc Methods: Many of IMP's internal procedures are realized using "ad hoc methods" and "hand-coded database entries," suggesting a lack of generalizable learning mechanisms.
  • Future Work: The authors suggest that the selection of unsolicited comments could be made more efficient and plausible with additional heuristics, particularly regarding the items' overall importance. They also note that the system's behavior can appear deceptive when its actual and projected biases diverge, highlighting the need for explicit and public investigation into the dialog motivations of such systems. Further research could explore more realistic ways people process probabilities and how to improve the system's response appropriateness.

Summary updated

BibTeX
@inproceedings{1983impression,
  title = {{Impression Monitoring in Evaluation-Oriented Dialog - The Role of the Listener's Assumed Expectations and Values in the Generation of Informative Statements}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {1983},
}

Loading related content...

Figure