TL;DR
This paper introduces a method for automatically generalizing translation examples in Example-Based Machine Translation (EBMT) systems by using bilingual word clustering. This approach significantly reduces the amount of parallel text needed for good translation coverage, making EBMT more practical for low-resource languages.
What Problem Does This Paper Address?
Example-Based Machine Translation (EBMT) systems rely on a large corpus of pre-translated sentence pairs to generate translations. For unrestricted texts, this often requires millions of words of parallel data, which can be difficult and expensive to obtain, especially for less common language pairs. While generalizing these examples (e.g., by creating templates or equivalence classes) can drastically reduce the required corpus size, manually creating these generalizations is labor-intensive. This paper addresses the challenge of automating this generalization process to make EBMT more accessible and efficient.
Methods
The core of the paper is a method for bilingual word clustering that leverages standard monolingual document clustering techniques.
- Monolingual Context Representation: For each unique word (or word pair in the bilingual context), a "pseudo-document" is created. This document consists of the words found in the surrounding context of that word's occurrences in the parallel corpus. These context words are weighted based on their proximity to the target word, with closer words having more influence. This creates a term vector for each word/pair.
- Bilingual Seeding: To inject bilingual information, the authors use a bilingual dictionary (derived from the corpus or external sources) to create initial "word pairs" (e.g., "cinq" - "five"). These pairs are treated as indivisible tokens.
- Context Generalization with Seeds: When processing contexts, if a word in the context corresponds to a word within a seeded equivalence class (e.g., "Friday" and "Monday" both belonging to a "days of the week" class), the name of the equivalence class is used instead of the original word. This helps generalize contexts even before explicit clustering.
- Clustering Algorithms: Both incremental group-average clustering and bottom-up agglomerative clustering algorithms were tested. The paper focuses on agglomerative clustering, specifically using maximal cosine similarity (single-link), as it produced subjectively better clusters.
- Thresholding: A frequency-dependent threshold is used to determine when to merge clusters. This accounts for the fact that high similarity between low-frequency words is more likely to be coincidental than between high-frequency words.
Datasets
- Training Corpus: A subset of the IBM Hansard corpus of Canadian parliamentary proceedings (approximately 1 million words, split between French and English).
- Test Text: Held-out Hansard text (approximately 45,000 words).
- Bilingual Dictionary: Used for initial seeding and word-level alignment. The ARTFL French-English dictionary was used, augmented by statistical methods and the ARTFL French-English dictionary itself.
Key Insights and Ideas
- Automated Generalization is Key: The paper demonstrates that automating the creation of equivalence classes through clustering can achieve most of the performance gains previously only possible with manual generalization and production rules.
- Bilingual Clustering is Necessary: Monolingual clustering isn't enough; injecting bilingual information (via seeded word pairs) is crucial for effective generalization in translation.
- Seeding Boosts Performance: Even a small set of manually-created "seed" equivalence classes significantly improves performance and further reduces the required training data. This is attributed to merging clusters that might otherwise remain distinct.
- Cluster Quality vs. Linguistic Purity: The paper highlights that clusters don't need to be linguistically pure (e.g., all conjunctions) to be useful. If words within a cluster allow for consistent translation patterns, they are valuable for EBMT.
- Phrases as Units: The method is extensible to phrases, not just single words, which would overcome a current limitation.
Key Results
- Reduced Corpus Size:
- Achieving 80% coverage of the test text required less than 300,000 words with manual generalization and approximately 300,000 words with automatic clustering. This is a reduction from 1.2 million words without generalization.
- Achieving 90% coverage required less than 500,000 words with manual generalization and less than 1.2 million words with automatic clustering, compared to 7 million words without generalization.
- Performance Gain: The automatic clustering approach provides a "considerable boost" to EBMT performance, reducing the required training text by a factor of four to five with little to no degradation in translation quality.
- Seeding Impact: Adding a small set of 425 seed pairs further reduced the required training text, largely by enabling more cluster merging.
- Efficiency: The clustering program is reasonably fast, with context vector creation at about 500,000 words per minute and agglomerative clustering of 17,527 distinct word pairs taking about 25 minutes.
Limitations and Future Work
- Iterative Clustering: An obvious extension is to use the output of a clustering run as the input seed for a second iteration. Preliminary experiments were inconclusive, suggesting the need for stricter thresholds in subsequent rounds.
- Advanced Clustering Algorithms: Exploring more sophisticated algorithms like k-means and deterministic annealing could yield better clusters at the cost of increased processing time.
- Phrase-Level Clustering: The current method is limited to single words. Extending it to cluster phrases would be a significant advancement.
- Grammar Generation from Clusters: The authors suggest foregoing traditional grammar rules and instead creating a grammar guided solely by automatically generated clusters. This, combined with automatically acquired parallel text, could lead to fully automated EBMT systems.
Summary updated