Lexical Selection in the Process of Language Generation

James Pustejovsky, Sergei NirenburgACL 1987

TL;DR

This paper argues that lexical selection is a crucial, often underestimated, component of natural language generation. The authors propose a novel classification of closed-class words and emphasize the role of discourse and focus information in guiding lexical choices, moving beyond purely grammatical constraints.

What Problem Does This Paper Address?

Computational linguists generally agree that natural language understanding requires robust lexical disambiguation. However, the inverse process, generation, has historically focused more on grammatical structure and control decisions, treating lexical selection as a secondary concern. This paper addresses the gap by arguing that principled lexical selection, informed by a deeper understanding of word types and contextual information, is essential for generating natural and coherent language. The authors highlight that existing generation systems often overlook the nuanced choices involved in selecting words, especially for closed-class items, and how these choices are influenced by factors beyond propositional content.

Methods

The paper proposes a more sophisticated approach to lexical selection in natural language generation. Key methods include:

  • Distinction between Open-Class and Closed-Class Items: Open-class items (nouns, verbs, adjectives) are treated as having independent semantic content, while closed-class items (determiners, prepositions, conjunctions) are seen as having procedural attachments to conceptual and discourse information.
  • Classification of Closed-Class Items: A novel distinction is introduced between Discourse-Oriented Closed-Class (DOCC) items and Proposition-Oriented Closed-Class (POCC) items. DOCC items (e.g., pronouns, determiners) rely heavily on discourse and contextual knowledge, while POCC items (e.g., primary verbs, some prepositions) are more tied to propositional content.
  • Focus-Driven Lexical Selection: The paper elevates the importance of "focus" (what is emphasized in a sentence) as a primary heuristic for controlling lexical selection, extending its influence beyond just anaphoric pronoun choices.
  • Concept Lexicon Design: A comprehensive design for a concept lexicon is suggested, combining a general semantic knowledge base with a generation-specific lexicon structured as discrimination nets. These nets use semantic and pragmatic tests to guide lexical choices.
  • Distributed Architecture: The authors advocate for a distributed generation architecture where lexical and grammatical decisions are not rigidly ordered, allowing for more flexible and opportunistic decision-making.

Datasets

Not discussed in the paper.

Key Insights and Ideas

  • Lexical Selection is Not a Side-Effect: The paper strongly argues that lexical selection is an active, constraining force in generation, not merely a consequence of grammatical decisions.
  • DOCC vs. POCC is Crucial: The distinction between discourse-oriented and proposition-oriented closed-class items provides a more nuanced way to understand their generation, highlighting their different dependencies on contextual versus propositional information.
  • Focus is a Powerful Heuristic: The emphasis on "focus" as a driver for lexical choice is a significant insight, suggesting that what is being emphasized in the discourse directly impacts word selection, even for non-discourse items.
  • Beyond Grammatical Constraints: The authors push for generation systems to integrate discourse and pragmatic knowledge more deeply into lexical selection, moving beyond systems that primarily rely on grammatical rules.

Key Results

The paper presents a conceptual framework and argues for its effectiveness rather than reporting quantitative results from a specific system implementation. The key "results" are the proposed distinctions and the emphasis on specific types of knowledge for lexical selection. The authors illustrate their points with examples, such as the choice between "buy" and "sell" based on focus, and the generation of pronouns based on discourse context.

Limitations and Future Work

  • System Implementation: While the paper outlines principles and a design, a full implementation and empirical evaluation of the proposed system are not detailed.
  • Completeness of DOCC/POCC: The paper mentions discussing other DOCC items (quantifiers, primary verbs) and other POCC items in a complete paper, suggesting the current discussion is not exhaustive.
  • Focus Representation: The precise mechanisms for representing and utilizing "focus" across various expression types are complex and would require further elaboration.
  • Future Work: The authors indicate that a complete paper will discuss the generation of other DOCC items and provide a detailed discussion of their distributed architecture. They also suggest that their approach is closer to McKeown's TEXT system in its use of discourse information to constrain generation, implying further exploration in this direction.

Summary updated

BibTeX
@inproceedings{pustejovsky1987lexical,
  author = {James Pustejovsky and Sergei Nirenburg},
  title = {{Lexical Selection in the Process of Language Generation}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {1987},
}
Figure