Abstract
Deirdre Hogan, Conor Cafferkey, Aoife Cahill, Josef van Genabith. Proceedings of the 2007 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning (EMNLP-CoNLL).
2007.
Introduction
Surface realization is the process of generating a fluent, grammatical sentence from an abstract syntactic representation. In this work, the authors focus on Lexical-Functional Grammar (LFG) "f-structures" as the input. While Probabilistic Context-Free Grammars (PCFGs) are commonly used for this task, they often fail due to strict independence assumptions. A PCFG makes local decisions without "remembering" the broader context, leading to errors in case marking—for instance, generating "her accepted" instead of "she accepted" because the model loses track of whether the pronoun is a subject or an object.
Furthermore, standard generators struggle with Multi-Word Units (MWUs) like "New York City." If these are not treated as atomic blocks, the generator may fragment them or produce incorrect word orders, reducing both fluency and coverage.
Methodology
The researchers propose two major enhancements to the baseline generative model:
1. History-Based Conditioning
To overcome the "short-term memory" of PCFGs, the model conditions its choices on the parent's grammatical function (GF). By knowing that a parent node is a "Subject," the generator can correctly select the nominative case for a pronoun. This approach effectively breaks the restrictive independence assumptions by providing the necessary structural context.
2. Multi-Word Unit (MWU) Integration
The authors explored three strategies for handling MWUs:
- Concatenation: Merging units (e.g., "New_York") into a single token during training and generation.
- Test-time Chunking: Merging units only in the input f-structure, leaving the underlying grammar unchanged.
- Internal Constraints: Adding a rule to the generation algorithm that penalizes any output that breaks the internal word order of a known named entity.
Datasets and Evaluation
The model was trained and tested on the Penn Wall Street Journal (WSJ) Treebank (Sections 02-21 for training, 23 for testing). To evaluate MWU handling, the authors utilized the BBN Pronoun Coreference and Entity Type Corpus as a gold standard for named entities, along with the Stanford Multi-word Expression List.
Results
The history-based (HB) model demonstrated significant improvements over the baseline on Section 23 of the WSJ treebank:
- Accuracy: The BLEU score increased from 66.52 to 68.82.
- Coverage: The system's ability to generate a valid output for a given input reached a near-perfect 99.96%, up from 98.18%.
- Significance: The HB model showed statistically significant gains even before MWU logic was added. The best performance was achieved using "Gold Standard" MWUs from the BBN corpus.
Conclusion
This research demonstrates that a well-informed generative model can rival more complex discriminative models. By simply "looking up" at the parent's role in a sentence, the system resolves most case-marking issues. Additionally, treating MWUs as "islands of certainty" reduces complexity and prevents the generator from producing fragmented or disordered phrases. Future work will focus on more sophisticated smoothing techniques to handle sparser data as conditioning context increases, as well as applying these methods to morphologically richer languages.
Summary updated