Abstract
Rashmi Gangadharaiah. Proceedings of 5th International Joint Conference on Natural Language Processing.
2011.
TL;DR
This paper introduces an automated method to identify and remove "redundant" source words—words necessary for grammar in one language but meaningless in another—to improve machine translation (MT). By pruning these asymmetries and using a lattice-based decoding approach, the researchers achieved a massive 5.29 BLEU point improvement in English-to-Hindi translation.
What Problem Does This Paper Address?
In Machine Translation, a common assumption is that every word in a source sentence has a partner in the target sentence. In reality, languages are asymmetrical. English uses "syntactic glue" like articles (the, a, an) and specific prepositions that often have no direct equivalent in languages like Hindi.
Standard alignment tools (like GIZA++) try to map these "extra" words to a special "NULL" token. However, when training data is scarce or noisy, the aligner gets confused. It might incorrectly map "the" to a random Hindi noun, polluting the translation dictionary and lowering overall quality. This paper tackles the challenge of automatically identifying which source words are essentially "noise" for a specific language pair and removing them to create cleaner alignments.
Methods
The authors propose a multi-stage pipeline to detect and handle redundant words without requiring manual linguistic rules or POS taggers:
- Scoring Candidates: Four statistical features derived from initial word alignments rank words for removal: Entropy, NULL Probability, Uniqueness, and Target Constraints (checking if a word sometimes has a valid translation).
- Constraint-Based Removal: Instead of "blindly" deleting every instance of a word like "the," they extract constraints from the lexical translation table. A word is considered a candidate for removal only in specific contexts where its translation probability falls below a threshold.
- Lattice Decoding: This is the core technical innovation for the testing phase. Rather than forcing a single deleted version of a sentence on the MT system, they provide a word lattice. This graph contains multiple possible paths—some containing the redundant words and some without them. The MT decoder then chooses the path that yields the highest statistical probability.
Datasets
The study focused on the English-Hindi language pair using:
- Darpa TIDES: A surprise language dataset from 2002.
- Internal News Corpus: Parallel news articles collected by IBM.
- Scale: 200,000 sentence pairs for training, a 500-sentence development set, and a 4,000-sentence test set.
- Alignment Evaluation: A small subset of 4,000 sentence pairs was manually hand-aligned to calculate F-scores for alignment quality.
Key Insights and Ideas
- Redundancy isn't just for Function Words: While "the" and "of" are obvious candidates, the system also identified certain content words as redundant depending on the noise in the dataset (e.g., words like "says" or "india" appearing as redundant due to specific alignment errors).
- The "NULL" Limitation: IBM alignment models are mathematically limited; they expect each source word to align to exactly one target word. By removing redundant source words, the authors effectively bypass this limitation, allowing the aligner to focus on meaningful correspondences.
- Lattices Over Hard Deletion: The decision to use a lattice during testing acknowledges that "redundancy" is probabilistic. By letting the decoder decide whether to keep or drop a word based on the context of the whole sentence, the system avoids the risks of "over-pruning."
Key Results
The results were statistically significant and quite high for the field of MT:
- BLEU Score Jump: The system improved from a baseline of 7.06 to 12.35 (4-gram BLEU), a gain of 5.29 points.
- Alignment Quality: The F-score for word alignment improved from 49.08 to 50.74.
- The "the" Effect: Interestingly, the authors found that "blindly" removing all instances of "the" actually decreased performance. Only when they used their constraint-based, lattice-driven approach did the scores climb, proving that context-aware removal is essential.
Limitations and Future Work
While the results are impressive, the paper notes a few areas for growth:
- One-Way Street: The current method only removes redundancy from the source language. Removing redundancy in the target language (and using post-processing to re-insert necessary grammatical markers) could further improve fluency.
- Language Specificity: The study is heavily focused on English-to-Indian languages. While the logic should hold for other asymmetrical pairs (like English-to-Japanese), it hasn't been tested there yet.
- Dependency on Initial Alignment: The method relies on an initial alignment from GIZA++. If the initial data is too poor, the features might not be reliable enough to identify the right candidates for removal.
Summary updated