Degree of Comparability: Enhancing Bilingual Lexical Extraction and Cross-Language Information Retrieval
Abstract
This paper investigates the comparability of documents within comparable corpora to improve the quality of extracted bilingual lexicons and the performance of cross-language information retrieval (CLIR) systems. We propose a novel approach that ensures a high degree of comparability and homogeneity within the corpus while preserving a significant portion of the original vocabulary. Our experiments demonstrate that the resulting bilingual lexicons are of higher quality than those obtained via previous methods and significantly enhance CLIR performance.
Introduction
In Natural Language Processing (NLP), parallel corpora—collections of direct translations—are the gold standard for training translation models. However, such resources are rare and expensive. Researchers often rely on comparable corpora, which consist of documents in two languages covering similar topics but not being direct translations (e.g., Wikipedia articles on "Climate Change" in English and French). The primary challenge lies in the "noise" or weak comparability of these datasets, where thematic mismatches hinder the extraction of accurate word equivalents.
Methodology
The authors address this by introducing a method centered on Agglomerative Hierarchical Clustering (CAA) to group documents by theme:
- Comparability Metric: A measure calculating the likelihood of finding a translation for a source word in the target vocabulary using an initial seed dictionary.
- Bilingual Clustering: Documents are organized into dendrograms to identify "deep" clusters with high thematic similarity.
- Penalty Function ($\phi$): To prevent lopsided clusters (e.g., 100 English documents matched with 1 French document), a penalty function ensures balanced language representation.
- External Enrichment: The algorithm can leverage a larger external corpus (like Wikipedia) to find better thematic matches for documents that are poorly represented in the original set.
Key Results
The proposed methods, particularly the $P^2$ approach involving external enrichment, yielded substantial improvements in lexicon quality. As shown in Table 1, the precision increased from 0.226 (baseline $P^0$) to 0.461 ($P^2$), representing a 104% relative improvement. Recall similarly doubled, moving from 0.103 to 0.212.
Further analysis by word frequency classes (Table 2) reveals that the improvements are consistent across low ($W_l$), medium ($W_m$), and high ($W_h$) frequency terms. The homogenization process ($P^{2'}$) alone provides a significant boost, which is then further amplified by the enrichment step ($P^2$).
In CLIR tasks, these extracted lexicons significantly improved Mean Average Precision (MAP). Adding extracted terms to a standard dictionary increased MAP from 0.375 to 0.391, a statistically significant gain.
Key Insights
- Homogeneity is Crucial: Narrowing the thematic focus makes the distributional hypothesis—that words in similar contexts are likely translations—far more effective.
- The "Core" Strategy: Extracting a highly comparable "core" from a noisy dataset produces higher quality results that outweigh the reduction in data volume.
- Resource Independence: The improvements in search performance suggest that the method captures general linguistic relationships rather than just dataset-specific patterns.
Conclusion
While the computational complexity of hierarchical clustering remains a challenge for web-scale applications, this research provides a robust framework for improving bilingual lexical extraction. Future work will focus on optimizing these algorithms for fully automatic construction of comparable collections from the web.
Summary updated