TL;DR
This paper introduces a novel method to automatically extract high-quality parallel fragments from noisy parallel corpora, a crucial step for improving machine translation systems. By leveraging existing word alignment models in a new way, they can identify and isolate useful translation units that were previously obscured by data imperfections, leading to significant improvements in translation quality.
What Problem Does This Paper Address?
The effectiveness of modern machine translation (MT) systems, particularly data-driven ones, heavily relies on the availability of large, high-quality parallel corpora (texts and their translations). However, acquiring such data is becoming increasingly difficult and expensive. Furthermore, much of the readily available data, especially from web scraping, is "noisy" – containing misalignments, irrelevant sentences, or other imperfections. These noisy alignments can severely degrade the performance of MT systems by introducing incorrect translation rules. Existing methods for extracting parallel data often struggle with this noise, and the authors aim to address this by developing a technique to robustly extract useful parallel fragments even from imperfect data.
Methods
The core of the paper's contribution lies in using a discriminatively trained word alignment model not just for alignment, but as a detector of noisy data. The authors employ a hierarchical alignment model that generates full derivation trees, representing a hierarchical partitioning of alignments. They then traverse these derivation trees top-down. The key idea is that segments with low alignment model scores are likely to be noisy or problematic.
They extract parallel fragments based on the following criteria:
- Candidate Constraint: The parent node in the derivation tree must have a score below a certain threshold ($\tau$), indicating a potentially problematic segment.
- Conservativity Constraint: The English span of the fragment must be longer than 3 words. This is because very small fragments are often extractable even from noisy data, and larger fragments are more likely to be affected by stray alignment links.
- Content Constraint: The fragment should not contain unaligned foreign words that are aligned to English words outside the fragment. This prevents the erroneous learning of translation rules that would delete or misplace words.
The threshold $\tau$ is set to capture the bottom 10% of segments by score, assuming these are the most likely to be noisy.
Datasets
The authors evaluate their method on large-scale Chinese-English and Arabic-English parallel corpora.
- Chinese-English: An initial corpus of 8.4M segments (210M English words, 193M Chinese words) was used. After extraction, the corpus grew to 9.2M segments (217M English words, 198M Chinese words).
- Arabic-English: An initial corpus of 9.0M segments (223M English words, 194M Arabic words) was used. After extraction, the corpus grew to 10M segments (233M English words, 202M Arabic words).
The paper mentions using LDC2005T10 and LDC2004T08 as sources for illustrative examples.
Key Insights and Ideas
- Alignment models as noise detectors: The most significant insight is repurposing an alignment model's "confusion" (low score) as a signal for noisy data, rather than just an indicator of poor alignment quality.
- Hierarchical structure for fragment extraction: The hierarchical nature of the alignment derivation trees provides a natural way to explore and partition potential parallel fragments at different granularities.
- Balancing fragment size and noise: The conservativity constraint (minimum fragment length) is crucial for ensuring that extracted fragments are substantial enough to be useful for rule extraction and are less likely to be trivially extractable by other means.
- Handling unaligned words: The content constraint is vital for preventing the introduction of spurious translation rules that might incorrectly handle words that are not part of the current parallel fragment.
Key Results
The paper demonstrates significant improvements in machine translation quality when using the extracted parallel fragments.
- Arabic-English: The system learned 68 million additional unique rules, resulting in a BLEU score increase from 50.0 to 50.4.
- Chinese-English: The system learned 38 million additional unique rules, leading to a BLEU score increase from 31.5 to 32.0.
These gains are statistically significant (p < 0.05 for Arabic-English and p < 0.01 for Chinese-English). The authors emphasize that these are not simply duplications of existing data but rather new, useful translation rules derived from the extracted fragments.
Limitations and Future Work
- Assumption of low score = noisy data: The core assumption is that low alignment scores directly correlate with noisy data. While the paper argues for this, it's a simplification, as low scores could also arise from genuine linguistic complexity or limitations of the alignment model itself.
- Threshold selection: The choice of the $\tau$ threshold (bottom 10%) is empirical. The paper suggests that experimenting with different ranges is possible but time-consuming.
- Future Work: The authors suggest that this method helps "squeeze more out of an already large, state-of-the-art MT system" by leveraging existing pipeline components. The work implicitly points towards further research in robust data processing for MT and other NLP tasks dealing with imperfect data.
Summary updated