Statistical query translation models for cross-language information retrieval

Jianfeng Gao, Jian-Yun Nie, Ming ZhouACM Trans. Asian Lang. Inf. Process 2006 · DOI

TL;DR

This paper introduces three statistical models for translating queries in cross-language information retrieval (CLIR) to improve translation accuracy by leveraging linguistic structures like noun phrases and dependency relations. By combining these models, the authors achieve significant improvements in CLIR performance compared to simple dictionary-based methods, demonstrating the value of incorporating syntactic information.

What Problem Does This Paper Address?

Cross-language information retrieval (CLIR) aims to allow users to search for information in languages they don't understand. A key challenge in CLIR is the "language mismatch" between a user's query and the documents, which are in a different language. While translating documents is an option, translating the query is generally more efficient. However, query translation itself is fraught with ambiguity, as single words can have multiple translations. This paper tackles the problem of resolving query translation ambiguities by developing statistical models that consider not just individual word translations but also their relationships within the query's linguistic structure.

Methods

The paper proposes three statistical query translation models:

  • Decaying Co-occurrence Model: This model treats a query as a "bag of words." It disambiguates translations by considering the co-occurrence of translated terms within a predefined window (like a sentence). It uses a "decaying factor" to give more weight to translations of words that are closer to each other in the query, assuming closer words have stronger relationships.
  • Noun Phrase (NP) Translation Model: This model identifies noun phrases within a query and translates them as single units. It assumes that the translation of a term within an NP depends only on other terms within the same NP. This leverages syntactic structure to improve translation accuracy for multi-word expressions.
  • Dependency Translation Model: This model goes further by identifying and translating "dependency triples" (e.g., verb-object, subject-verb) as units. It's based on the observation that dependency relations often remain consistent across languages. This model can handle discontinuous phrases, unlike the NP model.

These models are combined using a sequential combining approach, where each query term is assigned a single translation, and a parallel combining approach for CLIR evaluation, which allows for multiple translations per term to leverage query expansion effects.

Datasets

  • Training Data:
    • A Chinese newspaper corpus (approx. 80 million characters) for the co-occurrence model.
  • Penn Treebank for the NP translation model.
  • Unrelated English (Wall Street Journal, 1987-1992, 750MB) and Chinese (People's Daily, 1980-1998, 1200MB) corpora for the dependency translation model.
  • A word-aligned bilingual corpus (approx. 60K English-Chinese sentence pairs) for training translation templates.
  • Evaluation Data:
    • TREC-9 Chinese collections (for CLIR experiments).
  • TREC-5&6 corpus (for CLIR experiments).

Key Insights and Ideas

  • Linguistic Structure is Key: The core insight is that explicitly modeling linguistic structures (NPs and dependency relations) significantly improves query translation accuracy compared to treating queries as mere bags of words.
  • Dependency Relations are Robust: Dependency relations between words often translate consistently across languages, making them a powerful feature for disambiguation, even for discontinuous phrases.
  • Combining Models Yields Best Results: No single model is perfect. Combining the strengths of the co-occurrence, NP, and dependency models leads to the most robust and accurate query translations and, consequently, better CLIR performance.
  • Semantic Similarity Matters: Using cross-lingual semantic similarity (beyond strict dictionary translations) can help translate words or phrases not found in bilingual dictionaries, improving coverage.

Key Results

  • The three proposed models (COTM, NPTM, DPTM) all showed statistically significant improvements over a simple dictionary-based translation method (ST) for CLIR.
  • For short queries on TREC-9, COTM + NPTM achieved a 106.88% improvement over ST.
  • For long queries on TREC-9, COTM + NPTM + DPTM achieved a 64.92% improvement over ST.
  • The dependency translation model (Method C) achieved the highest accuracy (up to 81.0%) in translating verb-object dependency triples compared to dictionary-based (Method A) or co-occurrence-based (Method B) methods.
  • Combining all three models (COTM + NPTM + DPTM) consistently yielded the best CLIR performance across different datasets and query types.

Limitations and Future Work

  • Parser Robustness: The accuracy of detecting phrases and dependency triples is crucial, and errors in parsing can lead to translation errors. The authors suggest exploring partial parsers or grammatical relation detectors for improved robustness.
  • Data Sparsity: Training statistical models requires large amounts of data, particularly aligned bilingual corpora for translation templates. The lack of domain-specific datasets can hinder the translation of specialized terms. Future work could focus on automatically collecting bilingual corpora from the web.
  • Non-Compositional NPs: The NP translation model struggles with non-compositional NPs (e.g., political abbreviations) whose translations cannot be assembled from individual word translations. Improving the coverage of dictionaries or developing better methods for these types of phrases is needed.
  • Domain-Specific Terms: The dependency translation model, while powerful, can sometimes produce incorrect translations when frequent but semantically incorrect combinations of candidates are chosen.
  • Future Work: The authors suggest investigating more dependency relations, improving parser robustness, and exploring web-scale data collection for training.

Summary updated

BibTeX
@article{gao2006statistical,
  author = {Jianfeng Gao and Jian-Yun Nie and Ming Zhou},
  title = {{Statistical query translation models for cross-language information retrieval}},
  journal = {ACM Trans. Asian Lang. Inf. Process.},
  year = {2006},
  doi = {10.1145/1236181.1236184},
}
Figure