Identifying Parallel Documents from a Large Bilingual Collection of Texts: Application to Parallel Article Extraction in Wikipedia

Alexandre Patry, Philippe LanglaisBUCC @ ACL 2011 · OpenReview

Alexandre Patry and Philippe Langlais
Proceedings of the 4th Workshop on Building and Using Comparable Corpora, 2011

Abstract

This paper introduces PARADOCS, a language-neutral system designed to identify parallel (translated) document pairs within massive bilingual datasets. By leveraging "lightweight" features such as the sequence of numerical entities and unique words (hapax legomena), the system achieves high precision in identifying translations, significantly outperforming previous baselines while remaining computationally efficient.

Introduction

Machine Translation (MT) systems rely heavily on "parallel corpora"—sets of documents that are direct translations of each other. While the internet provides vast "comparable corpora" (documents on the same topic in different languages, such as Wikipedia), these are often not direct translations. Even when Wikipedia articles are explicitly linked across languages, the content can differ significantly. The challenge addressed here is to automatically extract the subset of documents that constitute true translations without relying on expensive, language-specific tools like bilingual dictionaries.

Methodology

PARADOCS utilizes a three-stage pipeline designed to be agnostic to specific language pairs:

  1. Candidate Search: Using the Lucene search engine, the system identifies the $N$ most likely matches for a source document. Efficiency is maintained by representing documents as sequences of hapax words (words appearing only once) or numerical entities (dates, prices, measurements).
  2. Classification: A classifier evaluates these candidates using three primary features: the Normalized Edit Distance between anchor sequences, the total Entity Count, and a Binary "Best Match" flag indicating if a pair has the lowest edit distance among candidates.
  3. Post-filtering: A logic step ensures one-to-one mapping between source and target documents, discarding potential duplicates to maintain high precision.

The system's reliance on numerical entities and hapax words acts as a structural "fingerprint," as these elements often remain stable and appear in the same relative order across translations.

Key Results

In controlled experiments using the Europarl (Version 5) corpus across 10 language pairs, PARADOCS demonstrated an average absolute gain of 13.6% in F-measure over the previous state-of-the-art baseline. The system proved robust across various languages, though performance is sensitive to document length.

Performance gain of PARADOCS across different language pairs and document lengths relative to the baseline.

As shown in the figure above, the gain is particularly pronounced for shorter documents where traditional methods struggle, although the system's absolute error rate does increase when fewer anchor entities are available (typically in documents under 32 sentences).

Key Insights and Analysis

One of the most significant findings is the inherent noise in existing cross-language links. A manual evaluation of 200 French-English Wikipedia article pairs revealed that only 14% are direct translations. Nearly half (46%) share a broad topic but are essentially unrelated in text.

Comparison of document types in raw Wikipedia links versus the PARADOCS-extracted subset.

When applied to the full Wikipedia dataset (over 537,000 pairs), PARADOCS identified over 44,000 parallel pairs. Manual verification confirmed that 80% of these identified pairs were either direct or "noisy" translations (mostly translations), demonstrating the system's ability to effectively filter high-quality data from noisy comparable sources.

Limitations and Future Work

  • Short Document Sensitivity: The reliance on unique markers means very short documents may lack sufficient "fingerprints" for reliable matching.
  • Filtering Strategy: The current dup filtering strategy is aggressive; a confidence-based selection could potentially improve recall.
  • Preprocessing: Wikipedia-specific elements like infoboxes and HTML markup can occasionally confound the classifier, suggesting that improved data cleaning could further enhance precision.

Summary updated

BibTeX
@inproceedings{patry2011identifying,
  author = {Alexandre Patry and Philippe Langlais},
  title = {{Identifying Parallel Documents from a Large Bilingual Collection of Texts: Application to Parallel Article Extraction in Wikipedia}},
  booktitle = {BUCC @ ACL},
  year = {2011},
  url = {https://openreview.net/forum?id=063zISXc8qp},
}
Figure