From Words to Corpora: Recognizing Translation

Noah A. SmithEMNLP 2002 · OpenReview

Abstract

This paper presents a technique for discovering translationally equivalent texts. It is comprised of the application of a matching algorithm at two different levels of analysis and a well-founded similarity score. This approach can be applied to any multilingual corpus using any kind of translation lexicon; it is therefore adaptable to varying levels of multilingual resource availability. Experimental results are shown on two tasks: a search for matching thirty-word segments in a corpus where some segments are mutual translations, and classification of candidate pairs of web pages that may or may not be translations of each other. The latter results compare competitively with previous, document-structure-based approaches to the same problem.

Introduction

The development of modern statistical machine translation (SMT) systems relies heavily on the availability of large-scale parallel corpora. However, high-quality parallel data is often a scarce resource. While the internet provides a vast potential source of such data, identifying true translation pairs within massive, unaligned document collections remains a significant challenge. Previous approaches, such as the STRAND system, have primarily utilized structural metadata (e.g., HTML tags) to identify parallel web pages. This paper proposes a robust, content-based method that identifies translations by analyzing word-level correspondences, making it applicable even in the absence of structural cues.

Methodology: The $tsim$ Similarity Score

The core of the proposed method is a two-level matching strategy. At the first level, the algorithm calculates a similarity score, $tsim$, between two documents $X$ and $Y$. By treating documents as "bags of words," the algorithm seeks the optimal one-to-one mapping between words in $X$ and their potential translations in $Y$, as defined by a translation lexicon.

This is formally modeled as a Maximum Weighted Bipartite Matching (MWBM) problem. The $tsim$ score is defined as the ratio of the weight of the maximum matching to the total possible weight, effectively measuring the degree of translational overlap.

Figure 1: An example of word-level alignment between English and French segments, illustrating the bipartite matching approach.

To improve computational efficiency, the author also introduces Competitive Linking (CL), a greedy approximation of MWBM. While MWBM has a complexity of $O(n^3)$, CL operates in $O(n \\log n)$ and provides comparable performance in many practical scenarios. Once $tsim$ scores are obtained for all candidate pairs, a second round of bipartite matching is performed at the document level to identify the global set of translation pairs across the corpus.

Experimental Evaluation

The effectiveness of the $tsim$ approach was evaluated on two distinct tasks:

  1. Hong Kong Hansard (English-Chinese): A controlled experiment using government transcripts to test precision and recall across different lexicon types and under varying levels of noise.
  2. STRAND Web Data (English-French): A real-world test on 325 candidate web page pairs to compare content-based recognition against structure-based methods.

Key Results and Insights

The experiments revealed several critical insights into the performance of the $tsim$ metric:

  • Lexicon Flexibility and Performance: The system is highly adaptable to the quality of the available translation lexicon. Performance was tested using a formal dictionary (DICT), an automatically induced translation model (TMTL), and a combined "Union" lexicon (UTL). As shown in Figure 3(a), the UTL lexicon yielded the best results, with MWBM achieving an F-score of 0.871.
  • Robustness to Noise: A key strength of the algorithm is its resilience in noisy environments. Figure 3(b) illustrates how precision remains high even as the ratio of true translations in the search space ($k/n$) decreases, provided an appropriate similarity threshold is maintained.
Figure 3: (a) Precision-Recall curves for MWBM and CL across different lexicons; (b) Performance under varying noise levels (k/n).
  • Content vs. Structure: On the STRAND web dataset, the $tsim$ classifier achieved a Cohen’s $\\kappa$ of 0.85, significantly outperforming the structure-based STRAND system, which scored 0.75. This demonstrates that content-based analysis can be more reliable than structural heuristics for identifying parallel web content.

Conclusion

This paper demonstrates that a content-based approach to translation recognition, grounded in bipartite matching, is both effective and versatile. By moving beyond structural metadata, the $tsim$ method enables the discovery of parallel text in diverse and noisy environments. Future work may involve iterative bootstrapping to grow lexicons and corpora simultaneously, as well as hybrid models that combine content-based and structure-based features for even greater accuracy.

Summary updated

Abstract

This paper presents a technique for discovering translationally equivalent texts. It is comprised of the application of a matching algorithm at two different levels of analysis and a well-founded similarity score. This approach can be applied to any multilingual corpus using any kind of translation lexicon; it is therefore adaptable to varying levels of multilingual resource availability. Experimental results are shown on two tasks: a search for matching thirty-word segments in a corpus where some segments are mutual translations, and classification of candidate pairs of web pages that may or may not be translations of each other. The latter results compare competitively with previous, document-structure-based approaches to the same problem.
BibTeX
@inproceedings{smith2002from,
  author = {Noah A. Smith},
  title = {{From Words to Corpora: Recognizing Translation}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2002},
  url = {https://openreview.net/forum?id=B1ZqIGzdWH},
}
Figure