Combining Statistical Translation Techniques for Cross-Language Information Retrieval

Ferhan Türe, Jimmy Lin, Douglas W. OardCOLING 2012 · OpenReview

Introduction

Cross-Language Information Retrieval (CLIR) allows users to search for documents in a foreign language using queries written in their native tongue. Historically, researchers have been forced to choose between two suboptimal approaches: simple token-to-token mapping using bilingual dictionaries, or treating Machine Translation (MT) as a "black box." While the former provides a diverse set of translation alternatives, it often lacks context, leading to noisy results. Conversely, standard MT systems provide contextually relevant translations but typically output only a single "best" result, which may fail to match the specific vocabulary used in the target document collection.

This paper proposes a middle ground: "peeking inside" the internal data structures of Statistical Machine Translation (SMT) systems. By extracting and combining context-independent word mappings with phrase-level and sentence-level evidence, the authors demonstrate a significant and consistent improvement in retrieval effectiveness across multiple language pairs.

Methodology: A Tri-Level Probabilistic Model

The core of this research is a unified probabilistic model that integrates three distinct scales of translation evidence through linear interpolation. These probabilities are then used to construct Probabilistic Structured Queries (PSQ), where each query term is replaced by a weighted set of potential translations for processing by a standard BM25 retrieval engine.

1. Token-based Evidence ($Pr_{token}$)

This represents the traditional baseline. It utilizes context-independent word alignments learned from large parallel corpora. While it offers broad coverage, it is prone to ambiguity (e.g., failing to distinguish between "bank" as a financial institution versus a riverbank).

2. Phrase-based Evidence ($Pr_{phrase}$)

To capture local context, the authors extract probabilities directly from the SMT system’s Synchronous Context-Free Grammar (SCFG). By analyzing the specific grammar rules applied to query phrases, this model creates a bias toward translations that are linguistically appropriate within a local window.

3. Query-based Evidence ($Pr_{nbest}$)

This model captures global, sentence-level context by reconstructing probabilities from the top $n$ full-sentence translations produced by the MT decoder. This ensures that the overall structure and intent of the query are respected in the translation candidates.

Experimental Setup

The approach was evaluated across three diverse language pairs to ensure generalizability:

  • English-Arabic: TREC 2002 (383k documents, 50 topics).
  • English-Chinese: NTCIR-8 ACLIA (388k documents, 50 topics).
  • English-French: CLEF 2006 (177k documents, 73 topics).

The SMT models were trained on substantial parallel corpora, including GALE 2010 (Arabic), FBIS (Chinese), and Europarl v7 (French).

Key Results and Insights

The combined interpolation model consistently outperformed both the token-based baseline and the standard "one-best" MT translation.

Language Baseline (Token) Combined Model (MAP)
Arabic 0.2712 0.3058
Chinese 0.1507 0.1916
French 0.2617 0.3084

Contextual Complementarity

The research highlights that these three models cover different scales of context. Word-based models provide necessary breadth, phrase-based models capture local idioms, and query-based models maintain sentence integrity. Interestingly, the optimal weighting of these sources varies by language; for instance, Arabic retrieval was nearly optimized by the phrase-based model alone, whereas Chinese and French required a more balanced combination of all three sources.

Efficiency vs. Effectiveness

While the combined model is the most effective, it introduces a 44% increase in query processing time compared to the baseline. However, the authors found that using the $n$-best approach with $n=1$ (the single best full-sentence translation) was actually faster than the token-based baseline while still providing superior retrieval performance, offering a compelling trade-off for real-time systems.

Conclusion and Future Work

By treating SMT systems as transparent sources of rich linguistic data rather than opaque black boxes, this paper provides a framework for more robust CLIR. The authors identify "spurious ambiguity"—where $n$-best lists contain redundant strings from different internal paths—as a challenge that their phrase-based grammar approach helps mitigate.

Future research directions include developing automated, robust methods for tuning interpolation weights across different languages and exploring the application of these context-sensitive techniques to document-side translation, which could further enhance accuracy at the cost of higher offline computational requirements.

Summary updated

BibTeX
@inproceedings{ture2012combining,
  author = {Ferhan Türe and Jimmy Lin and Douglas W. Oard},
  title = {{Combining Statistical Translation Techniques for Cross-Language Information Retrieval}},
  booktitle = {International Conference on Computational Linguistics},
  year = {2012},
  url = {https://openreview.net/forum?id=tVm4wR4RQ0},
}
Figure