Effects of Empty Categories on Machine Translation

Tagyoung Chung, Daniel GildeaEMNLP 2010 · OpenReview

Abstract: We examine effects that empty categories have on machine translation. Empty categories are elements in parse trees that lack corresponding overt surface forms (words) such as dropped pronouns and markers for control constructions. We start by training machine translation systems with manually inserted empty elements. We find that inclusion of some empty categories in training data improves the translation result. We expand the experiment by automatically inserting these elements into a larger data set using various methods and training on the modified corpus. We show that even when automatic prediction of null elements is not highly accurate, it nevertheless improves the end translation result.

Introduction

In languages like Chinese, Japanese, and Korean, pronouns are frequently omitted when they can be inferred from context—a phenomenon known as "pro-drop." While humans fill in these blanks effortlessly, Machine Translation (MT) systems struggle. When translating into English—a language where pronouns are mandatory—the system has to generate words "from nothing." Standard phrase-based MT models often fail to produce the correct subject or object, leading to ungrammatical or nonsensical output. This research investigates whether explicitly identifying and re-inserting these "ghost words" (empty categories) into the source text before translation can help the model learn better alignments and produce more accurate English.

Mapping between a parse tree with an empty category (*pro*) and its surface structure.

Methodology

The study explored three distinct strategies for automatically "recovering" these missing elements:

  1. Pattern Matching: This method looks for specific "holes" in a sentence's parse tree. If a specific structural fragment (like a verb phrase missing a subject) is found, the system inserts an empty category marker.
  2. Conditional Random Fields (CRF): A sequence labeling approach that examines word boundaries and uses local context (words and parts-of-speech) to decide whether to insert a dropped pronoun (*pro*) or a control marker (*PRO*).
  3. Latent-Variable Parsing: This method modifies a parser to treat empty categories as special annotations on the grammar's non-terminal symbols. The parser essentially learns to "see" the missing nodes as part of its normal structural analysis.
Encoding empty categories into non-terminal labels for latent-variable parsing.

Experimental Setup

The study utilized several key linguistic resources:

  • Penn Chinese Treebank (CTB): Used for initial experiments and training the recovery models.
  • Korean Treebank: Used to validate the approach on a second pro-drop language.
  • FBIS Newswire Data: A larger bilingual Chinese-English corpus (approx. 60K sentences) used for final translation evaluation.
  • NIST 2002 MT Evaluation Set: The benchmark for testing translation quality.

Key Insights

  • Category Impact: While many types of empty categories exist, dropped pronouns (*pro*) and control structures (*PRO*) provide the most significant boost to translation quality.
  • Structure vs. Context: Structural patterns were best at finding *PRO* (tied to specific grammatical rules), while local word context (CRF) was superior at identifying *pro* (dependent on discourse context).
  • Robustness to Error: Recovery doesn't have to be perfect. Even with relatively low F1 scores (0.44 to 0.63) for automatic insertion, the "noisy" markers provided enough signal to significantly improve the final English output.

Results

On small, manually annotated datasets, adding empty categories improved Chinese-to-English translation by 1.17 BLEU points and Korean-to-English by 0.27 points. In large-scale tests using CRF-based automatic insertion on the 60K-sentence FBIS dataset, the BLEU score rose from 23.73 to 24.69 (+0.96 points), a statistically significant improvement. Qualitative analysis showed systems trained with empty categories were better at producing "to-infinitive" forms in English and correctly identifying sentence subjects.

Conclusion and Future Work

The research demonstrates that even imperfect linguistic reconstruction of empty categories is better than translating from a "hollow" sentence. Future work could involve integrating recovery directly into the translation decoder or developing more sophisticated machine learning models to resolve conflicts between competing empty category predictions.

Summary updated

Abstract

We examine effects that empty categories have on machine translation. Empty categories are elements in parse trees that lack corresponding overt surface forms (words) such as dropped pronouns and markers for control constructions. We start by training machine translation systems with manually inserted empty elements. We find that inclusion of some empty categories in training data improves the translation result. We expand the experiment by automatically inserting these elements into a larger data set using various methods and training on the modified corpus. We show that even when automatic prediction of null elements is not highly accurate, it nevertheless improves the end translation result.
BibTeX
@inproceedings{chung2010effects,
  author = {Tagyoung Chung and Daniel Gildea},
  title = {{Effects of Empty Categories on Machine Translation}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2010},
  url = {https://openreview.net/forum?id=SyZj4zfubH},
}
Figure