Controlling Lexical Substitution in Computer Text Generation

Robert GranvilleACL 1984

TL;DR

This paper introduces "Paul," a computer text generation system that controls lexical substitutions (like using pronouns or more general terms) to create cohesive and understandable text. By analyzing the "strength of antecedence recovery" of these substitutions and the "strength of potential antecedence" of text elements, Paul deterministically selects the most appropriate way to refer back to previously mentioned concepts, avoiding ambiguity and unnatural repetition.

What Problem Does This Paper Address?

Generating natural-sounding text requires more than just stringing sentences together; it needs cohesion – a linguistic glue that ties sentences into a unified whole. A key aspect of cohesion is how we refer back to things already mentioned. Simply repeating words leads to monotonous text, while uncontrolled use of referring expressions like pronouns or superordinates (e.g., "animal" for "dog") can lead to confusion and ambiguity. This paper tackles the challenge of controlling these lexical substitutions in computer text generation to ensure clarity and naturalness, moving beyond simply avoiding repetition to actively managing how information is presented.

Methods

The core of Paul's approach lies in its sophisticated control over lexical substitutions. It focuses on three main types:

  • Pronominalization: Using pronouns (he, she, it, they).
  • Superordinate Substitution: Replacing a term with a more general one (e.g., "mammal" for "possum").
  • Definite Noun Phrase Reiteration: Repeating a noun phrase with a definite article ("the") to signal it's already known.

Paul's control mechanism is based on two key concepts:

  1. Strength of Antecedence Recovery: This measures how easily a reader can identify the original element being referred to by a cohesive device. Definite noun phrases have the highest strength (direct repetition), followed by superordinates (requiring tracing up a hierarchy), and then pronouns (lowest strength, as they carry little semantic information).
  2. Strength of Potential Antecedence: This assesses how easily an element in the text can be identified as a referent, based on its current and previous usage, syntactic role, and proximity to the current mention. The paper defines five classes for this, with Class I being the strongest (most easily identifiable).

Paul then maps these strengths: elements with high potential antecedence (Class I) are best referred to with weaker cohesive devices like pronouns, while elements with lower potential antecedence (Class IV, V) require stronger devices like definite noun phrases. Superordinate substitution is used for intermediate cases. The system also handles potential ambiguity in superordinate substitution by using features like color or size as modifiers.

Datasets

Not discussed in the paper. The paper uses a sample hierarchy and a generated story based on "Wait Kelly's Pogo comic strip" for illustrative purposes, but no formal datasets are mentioned for training or evaluation.

Key Insights and Ideas

  • Cohesion is not just about avoiding repetition, but about managing information flow: The paper highlights that effective text generation requires actively controlling how references are made to ensure clarity and guide the reader.
  • A deterministic approach to lexical substitution is possible and necessary: By quantifying the "strength" of both the referring expression and the referent, Paul can make principled, non-arbitrary choices about which substitution to use.
  • The "strength" of a reference is a dual concept: It depends on both the referring expression's inherent clarity (antecedence recovery) and the referent's prominence in the discourse (potential antecedence).
  • Superordinate substitution needs careful disambiguation: Simply using a general term can be ambiguous. Paul's use of features like color or size to modify superordinates is a crucial step in making them unambiguous.
  • Pronouns are the "weakest link" in cohesion: They offer the least semantic information, making them the most prone to misinterpretation if not used judiciously with highly prominent antecedents.

Key Results

The paper demonstrates Paul's ability to generate cohesive text through a specific example. It contrasts the output of a system with no lexical substitutions (repetitive and unnatural) and a system with uncontrolled pronominalization (confusing and nonsensical) with the output of Paul. The generated text for the "Pogo" story is presented as an example of controlled, understandable, and cohesive output, though specific quantitative metrics are not provided.

Limitations and Future Work

  • Ambiguity in Superordinate Substitution: While Paul has mechanisms to disambiguate superordinate substitutions using features, the paper acknowledges that finding a suitable disambiguating feature can be challenging.
  • Focus Algorithm: The paper mentions using Sidner's algorithm for identifying the text's focus but doesn't detail its implementation or its impact on the strength of potential antecedence.
  • Limited Scope of Lexical Substitutions: Paul currently handles only three types of lexical substitution. Future work could explore other cohesive devices.
  • No Quantitative Evaluation: The paper relies on qualitative examples rather than quantitative metrics to demonstrate Paul's effectiveness.

The paper suggests that the controlled approach to lexical substitution is a significant step towards generating truly coherent and natural-sounding computer-generated text.

Summary updated

BibTeX
@inproceedings{granville1984controlling,
  author = {Robert Granville},
  title = {{Controlling Lexical Substitution in Computer Text Generation}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {1984},
}
Figure