TL;DR
This paper introduces a novel approach to Cross-Lingual Information Retrieval (CLIR) by integrating retrieval directly into the decoding process of Statistical Machine Translation (SMT). By treating document ranking as a constrained decoding problem, they achieve significant improvements over existing methods on large-scale patent and Wikipedia datasets.
What Problem Does This Paper Address?
Cross-Lingual Information Retrieval (CLIR) aims to find relevant documents in a target language when the query is in a different source language. Traditional CLIR methods often rely on a pipeline: first, translate the query using SMT, then perform monolingual retrieval. While effective, this pipeline approach has limitations. SMT systems are typically optimized for producing fluent human translations, not necessarily for retrieval tasks, which often use bag-of-words representations and may not require grammatical fluency. This can lead to a loss of structural information and suboptimal lexical choices for retrieval. Existing attempts to bridge this gap either re-rank retrieved documents or use simplified query representations, but this paper proposes a more direct integration.
Methods
The core innovation is Bag-of-Words Forced Decoding (BOW-FD). Instead of translating the query and then searching, this method uses an SMT decoder in a "forced decoding" mode. The decoder is tasked with producing a bag-of-words representation of target documents that are likely to be relevant to the source query. This is achieved by augmenting the standard SMT log-linear model with retrieval-specific features. These features are designed to decompose over partial translation hypotheses, allowing for efficient decoding using dynamic programming on hypergraphs. Essentially, the SMT decoder is guided not only by translation fluency but also by how well the partial translation matches the content of a document. The model is trained using a learning-to-rank objective to jointly optimize both translation and retrieval aspects.
Datasets
The paper evaluates their approach on two large-scale CLIR tasks:
- German-English Wikipedia cross-lingual article retrieval: This dataset is a standard benchmark for CLIR.
- Japanese-English patent abstracts search: This dataset is used for patent prior art search.
The authors utilized existing SMT systems and evaluation scores from prior work (Schamoni et al., 2014) for these datasets.
Key Insights and Ideas
- Reversing the Pipeline: The paper successfully demonstrates that shifting the focus from retrieval to translation within the decoding process can be highly effective for CLIR.
- Joint Optimization is Key: Integrating translation and retrieval models directly, rather than in a pipeline, leads to significant performance gains. This suggests that optimizing for both aspects simultaneously is crucial for powerful CLIR models.
- Language Models are Crucial for Retrieval: Contrary to some prior claims, the authors found that removing language model information significantly degrades retrieval performance, especially for precision-oriented metrics. This highlights the dual role of LMs in ensuring fluency and selecting appropriate terms for retrieval.
- Forced Decoding for Retrieval: The concept of "forced decoding" is adapted from its traditional use (matching a reference translation) to a new context: forcing the decoder to produce a bag-of-words representation of relevant documents.
Key Results
- BOW-FD significantly outperforms state-of-the-art baselines (Direct Translation and Probabilistic Structured Queries) by over 2 points in MAP, NDCG, and PRES scores on both Wikipedia and patent datasets.
- Learning-to-rank (LTR) further improves performance, especially for recall-oriented metrics like PRES.
- Removing the language model from SMT-based models resulted in substantial drops in performance (e.g., average drop of 6 points in MAP and NDCG for Wikipedia, and over 11 points for patents).
Limitations and Future Work
- Efficiency in Online Applications: While the approach shows strong results, the inherent complexity of SMT decoding can be prohibitive for online applications. The authors suggest investigating the incorporation of inverted indexes for improved efficiency in future work.
- Multi-Sentence Queries: For patent search, which often involves multi-sentence queries, the paper uses a product-of-experts approach to combine sentence-wise rankings. Further exploration of how to best handle complex query structures could be beneficial.
Summary updated