Abstract
Min-Yen Kan, Kathleen R. McKeown. Proceedings of the International Natural Language Generation Conference.
2002.
Introduction
Traditional Natural Language Generation (NLG) systems have historically relied on extensive manual labor. Human experts were required to code complex rules for discourse, semantics, and grammar, making the adaptation of summarizers to new domains—such as medical literature or consumer reviews—prohibitively expensive and time-consuming. This paper addresses the "knowledge acquisition bottleneck" by proposing a framework that uses machine learning to induce generation rules from a small set of semantically annotated examples. The focus is on indicative summaries, which describe the contents of a document (like library catalog entries) rather than replacing the document itself.
The Acquisition Framework
The proposed system moves away from hand-crafted templates by learning both content planning and surface realization from a corpus. The acquisition process is divided into two primary modules:
- Content Planner Acquisition: This module determines which "predicates" (information types like Author, Topic, or Audience) are likely to appear and in what order. By calculating bigram statistics from annotated entries, the system identifies optimal sequences, using backoff schemes to handle long-distance relationships between sentences.
- Surface Realizer Acquisition: This module learns to transform data attributes into natural language. It distinguishes between the "attribute value" (the specific data) and the "associated text" (the linguistic frame).
Generation and Realization
To generate a summary, the system utilizes the knowledge bases (KBs) built during the acquisition phase. A lexical dependency parser identifies frequent "head-child" relationships (e.g., include $\rightarrow$ index) to separate data from boilerplate text. Furthermore, the system employs the RIPPER decision tree learner to predict stylistic features—such as sentence length or the use of passive voice—based on the surrounding linguistic context. This ensures that the phrasing of a sentence is sensitive to its neighbors, maintaining a natural flow.
Experimental Results
The system was trained on a primary corpus of 2,000 bibliography entries, with a 5% subset manually annotated with 24 semantic predicates. Key results include:
- Predicate Identification: Accuracy improved from a 0% baseline to 66-67% using features like word sets and position.
- Attribute Identification: The system achieved 94.8% precision in distinguishing attribute values from associated text.
- Stylistic Predictability: Stylistic features were frequently selected by the machine learner, confirming that style can be predicted from context.
Conclusion
While the system demonstrated high precision in attribute identification and significant improvements in semantic tagging, challenges remain regarding data sparseness for rare predicates. Future work involves expanding the annotated corpus and conducting end-to-end human evaluations of the generated summaries. This research marks a significant step toward automating the creation of domain-specific summarization systems.
Summary updated