Alexandre Patry and Philippe Langlais
Proceedings of the 4th Workshop on Building and Using Comparable Corpora, 2011
Abstract
This paper introduces PARADOCS, a language-neutral system designed to identify parallel (translated) document pairs within massive bilingual datasets. By leveraging "lightweight" features such as the sequence of numerical entities and unique words (hapax legomena), the system achieves high precision in identifying translations, significantly outperforming previous baselines while remaining computationally efficient.
Introduction
Machine Translation (MT) systems rely heavily on "parallel corpora"—sets of documents that are direct translations of each other. While the internet provides vast "comparable corpora" (documents on the same topic in different languages, such as Wikipedia), these are often not direct translations. Even when Wikipedia articles are explicitly linked across languages, the content can differ significantly. The challenge addressed here is to automatically extract the subset of documents that constitute true translations without relying on expensive, language-specific tools like bilingual dictionaries.
Methodology
PARADOCS utilizes a three-stage pipeline designed to be agnostic to specific language pairs:
- Candidate Search: Using the Lucene search engine, the system identifies the $N$ most likely matches for a source document. Efficiency is maintained by representing documents as sequences of hapax words (words appearing only once) or numerical entities (dates, prices, measurements).
- Classification: A classifier evaluates these candidates using three primary features: the Normalized Edit Distance between anchor sequences, the total Entity Count, and a Binary "Best Match" flag indicating if a pair has the lowest edit distance among candidates.
- Post-filtering: A logic step ensures one-to-one mapping between source and target documents, discarding potential duplicates to maintain high precision.
The system's reliance on numerical entities and hapax words acts as a structural "fingerprint," as these elements often remain stable and appear in the same relative order across translations.
Key Results
In controlled experiments using the Europarl (Version 5) corpus across 10 language pairs, PARADOCS demonstrated an average absolute gain of 13.6% in F-measure over the previous state-of-the-art baseline. The system proved robust across various languages, though performance is sensitive to document length.
As shown in the figure above, the gain is particularly pronounced for shorter documents where traditional methods struggle, although the system's absolute error rate does increase when fewer anchor entities are available (typically in documents under 32 sentences).
Key Insights and Analysis
One of the most significant findings is the inherent noise in existing cross-language links. A manual evaluation of 200 French-English Wikipedia article pairs revealed that only 14% are direct translations. Nearly half (46%) share a broad topic but are essentially unrelated in text.
When applied to the full Wikipedia dataset (over 537,000 pairs), PARADOCS identified over 44,000 parallel pairs. Manual verification confirmed that 80% of these identified pairs were either direct or "noisy" translations (mostly translations), demonstrating the system's ability to effectively filter high-quality data from noisy comparable sources.
Limitations and Future Work
- Short Document Sensitivity: The reliance on unique markers means very short documents may lack sufficient "fingerprints" for reliable matching.
- Filtering Strategy: The current
dupfiltering strategy is aggressive; a confidence-based selection could potentially improve recall. - Preprocessing: Wikipedia-specific elements like infoboxes and HTML markup can occasionally confound the classifier, suggesting that improved data cleaning could further enhance precision.
Summary updated