What Not to Say

Jan FornellACL 1984

This paper explores the challenges of creating natural-sounding text generation systems, arguing that current models often fail due to a neglect of pragmatic factors in communication. The authors introduce "Commentator," a system designed to study these issues by dynamically describing events and generating comments, and propose a new direction for research based on the concept of unification.

TL;DR

This paper tackles the problem of verbosity and unnaturalness in computational text generation by focusing on pragmatic factors like relevance and context. It introduces "Commentator," a system that generates dynamic, context-aware descriptions, and suggests that a deeper understanding of human discourse requires moving beyond simple rule-based generation towards more interactive and context-sensitive models.

What Problem Does This Paper Address?

Most text production and language generation systems tend to be overly verbose and unnatural. This is often because they focus on generating grammatically correct sentences without considering the pragmatic aspects of communication – essentially, why and how something should be said. The authors argue that this neglect of pragmatic factors, such as relevance, context, and the listener's presumed knowledge, leads to systems that state the obvious or fail to convey meaning effectively. They aim to explore how to build systems that can generate more natural and contextually appropriate language, moving beyond simply stating facts to understanding and conveying intent.

Methods

The paper describes two versions of a system called "Commentator." The original version was written in BASIC and acted as a "linguistic research tool" to study various levels of language generation, from raw data to coherent text. It dynamically described the movements of objects on a screen and generated comments in English and Swedish. A key strategy employed was to comment only on changes in the scene, avoiding redundant statements. Another relevance strategy was to focus on nearness, commenting on objects in relation to those that are physically close.

A new, modular version is being implemented in Prolog, aiming to develop a highly interactive model called "CONVERSATOR" to study human discourse. This new model intends to incorporate interactive components and explore properties of human discourse beyond simple monologue generation.

Datasets

Not discussed in the paper. The system operates on simulated "raw data" like object coordinates on a computer screen.

Key Insights and Ideas

  • Pragmatic Determinism: Systems should be "pragmatically deterministic" in the sense that they only investigate states that might lead to an expressible comment, rather than generating all possible outputs and then selecting.
  • Focus on Change: Commenting only on changes in a dynamic scene is a crucial strategy to avoid verbosity and redundancy.
  • Nearness as a Relevance Criterion: Commenting on objects in relation to nearby objects is more natural than referencing distant ones, even if both statements are factually true.
  • The Importance of "Why Talk?": To overcome the limitations of current text generation systems, it's essential to ask why we communicate, considering factors beyond just informative content, such as ritualized behavior and prompted speech.
  • Understanding as Creative Construction: Natural language understanding is not just decoding but a creative process of constructing a context where an utterance fits, involving linguistic, extra-linguistic, and world knowledge.
  • Unification as a Promising Concept: The paper suggests that the concept of unification, currently used in syntactic theories, could be a powerful tool for handling semantics, discourse structure, and world knowledge in computational pragmatics.

Key Results

The paper doesn't present quantitative results in terms of accuracy or performance metrics. Instead, it focuses on qualitative insights derived from the design and behavior of the "Commentator" system. The key "results" are the identified strategies for generating more natural language:

  • Commenting on changes leads to less verbose output.
  • Prioritizing nearby objects for commentary improves relevance.
  • The system's ability to generate contextually relevant comments, even with limited input, demonstrates the potential of pragmatic considerations.

Limitations and Future Work

Limitations: * The original Commentator model is "narrow and vertical," meaning it focuses on a single level of representation rather than integrating multiple levels comprehensively. * The system cannot truly know the intentions of the objects it describes; it can only make guesses based on observed behavior. Any presuppositions or implicatures are those of the system's beliefs, not the actual subject's. * The paper acknowledges that current approaches to understanding particularized conversational implicatures are "entirely untreatable." * A significant limitation for text generation systems is the lack of world knowledge, leading to outputs that are "highly artificial" and akin to phrase-book translations.

Future Work: * Developing a new, highly interactive model called "CONVERSATOR" to study human discourse properties. * Exploring the concept of unification as a mechanism for handling semantics, discourse structure, and world knowledge. * Investigating questions like "When does one initiate a conversation?" and "When does one change topic?" which require significant world and listener knowledge. * Connecting the system to sophisticated speech synthesis and machine translation facilities.

Summary updated

BibTeX
@inproceedings{fornell1984what,
  author = {Jan Fornell},
  title = {{What Not to Say}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {1984},
}
Figure