A Bootstrapping Method for Extracting Bilingual Text Pairs

Hiroshi Masuichi, Raymond Flournoy, Stefan Kaufmann, Stanley PetersCOLING 2000

TL;DR

This paper introduces a bootstrapping method to automatically extract bilingual text pairs from comparable corpora, which are more readily available than parallel corpora. By iteratively refining a small initial parallel corpus, their approach significantly improves the precision and recall of extracted translation pairs, making them valuable for training cross-language information retrieval (CLIR) systems.

What Problem Does This Paper Address?

Cross-language information retrieval (CLIR) systems often rely on parallel corpora (texts that are direct translations of each other) for training. However, these parallel corpora are scarce, small, and often domain-specific. Comparable corpora, which contain texts on the same topic but are not direct translations, are much more abundant. The challenge is to effectively extract accurate bilingual text pairs from these comparable corpora to serve as training data for CLIR, bridging the gap between readily available resources and the need for high-quality training data.

Methods

The core of the proposed method is a bootstrapping approach applied to an existing corpus-based CLIR technique called Information Mapping. Information Mapping uses a vector space model derived from word co-occurrence statistics in a training corpus to represent queries and documents in a language-independent space.

The bootstrapping process works as follows:

  1. Initialization: Start with a very small, manually created parallel corpus.
  2. Initial Retrieval: Use this small corpus to train an initial Information Mapping model. Then, use this model to retrieve bilingual text pairs from a larger comparable corpus. The retrieval is based on finding pairs where the vector representations of the English and Japanese texts have high proximity (cosine similarity). These are termed "mutual-proximity pairs."
  3. Refinement and Iteration: Select a small number of the most "reliable" mutual-proximity pairs (those with the highest proximity scores) and add them to the initial parallel corpus.
  4. Growth: Re-train the Information Mapping model with the expanded corpus and repeat the retrieval process. This iterative cycle of retrieval, selection, and expansion allows the system to gradually grow a larger and more accurate parallel corpus. The authors emphasize concatenating only a small number of reliable pairs in early stages to mitigate the sensitivity to initial errors.

Datasets

The experiments were conducted using an English-Japanese bilingual patent text corpus.

  • Pseudo Comparable Corpus: 1000 English-Japanese patent pairs were created. For each Japanese patent, its human translation existed, making it an "ideal bilingual comparable corpus."
  • Initial Parallel Corpus: 100 English-Japanese patent pairs were used as the initial training data.
  • Simulated Incomplete Corpora: To test robustness, the authors created corpora where half (50%) or 80% of the Japanese patent texts were replaced with irrelevant texts, simulating a more realistic, noisy comparable corpus.

Key Insights and Ideas

  • Bootstrapping for Data Augmentation: The core idea that a bootstrapping mechanism can effectively "grow" a parallel corpus from a comparable one is a significant contribution. This addresses the scarcity of parallel data.
  • Reliability-Based Selection: The strategy of selecting only the most "reliable" mutual-proximity pairs in early bootstrapping stages is crucial for preventing error propagation and ensuring the quality of the growing corpus.
  • Value of Extracted Pairs: The extracted bilingual text pairs are not just for creating a larger corpus but are shown to be useful for improving the performance of the original CLIR method when used as training data.
  • Robustness to Noise: While performance degrades with noisier corpora (50% or 80% irrelevant texts), the bootstrapping method still significantly outperforms the baseline CLIR method, demonstrating its resilience.

Key Results

  • Precision and Recall Improvement: The bootstrapping method consistently and significantly improved both precision and recall compared to the normal Information Mapping method (without bootstrapping).
  • In tests with an "ideal" comparable corpus, the bootstrapping method achieved a recall of 72.1% and a precision of 99.2%, compared to the normal method's 25.8% recall and 75.7% precision.
  • Even with 80% irrelevant texts, the bootstrapping method achieved 52.5% precision and 58.0% recall, while the normal method achieved 23.4% precision and 27.5% recall.
  • Gradual Improvement: Figures 2 and 4 show that precision and recall generally increased steadily over the 100 bootstrapping stages.
  • Training Data Efficacy: The text pairs extracted by bootstrapping were shown to be highly effective as training data for the normal Information Mapping method, bringing its performance close to the theoretical ceiling (using all correct translation pairs).

Limitations and Future Work

  • Sensitivity to Initial Pairs: The paper acknowledges that the bootstrapping method is highly sensitive to the accuracy of the text pairs obtained in the early stages. The proposed solution of gradually increasing the number of concatenated pairs mitigates this, but it remains a potential weakness.
  • Artificial Corpora: The experiments were conducted on artificial corpora (patent texts) and simulated noise. The performance on real-world, diverse comparable corpora might differ.
  • Domain Specificity: While the method aims to be general, its effectiveness might vary across different domains.
  • Future Work: The authors suggest that the extracted translation pairs could be useful for improving CLIR results, implying further research into how best to integrate them into existing CLIR frameworks. They also implicitly suggest exploring more sophisticated methods for selecting "reliable" pairs.

Summary updated

BibTeX
@inproceedings{masuichi2000bootstrapping,
  author = {Hiroshi Masuichi and Raymond Flournoy and Stefan Kaufmann and Stanley Peters},
  title = {{A Bootstrapping Method for Extracting Bilingual Text Pairs}},
  booktitle = {International Conference on Computational Linguistics},
  year = {2000},
}
Figure