IR like a SIR: Sense-enhanced Information Retrieval for Multiple Languages

Rexhina Blloshmi, Tommaso Pasini, Niccolò Campolungo, Somnath Banerjee, Roberto Navigli, Gabriella PasiEMNLP 2021 · DOI

Abstract: Sense-enhanced Information Retrieval (SIR) is a framework designed to address query ambiguity and the scarcity of multilingual training data in neural search. By expanding short queries with word sense definitions (glosses) from a multilingual knowledge base, SIR provides necessary context for transformer-based rankers. This approach enables effective zero-shot cross-lingual retrieval, where a model trained solely on English data outperforms state-of-the-art rankers in multiple other languages.

1. Introduction

Information Retrieval (IR) has transitioned from keyword-based methods like BM25 to neural ranking models such as BERT. Despite these advances, two primary challenges persist: the context gap in short, ambiguous search queries and the lack of large-scale labeled datasets for non-English languages. Neural models often struggle with the brevity of typical search queries, which lack the syntactic structure of the full sentences seen during pre-training. Furthermore, the "data drought" in many languages hinders the development of high-performance local search engines.

Query formation process illustrating the combination of title and description fields.

2. The SIR Framework

The SIR framework addresses these issues by splitting the retrieval process into two distinct modules: a Query Expander and a Document Ranker.

2.1 Query Expander

The Query Expander performs Word Sense Disambiguation (WSD) to enrich the query. It utilizes ARES embeddings—specialized vectors representing word meanings—to identify the most relevant "senses" for query terms. It then retrieves corresponding glosses (definitions) from BabelNet, a massive multilingual semantic network. These definitions are prepended to the original query, transforming keywords into descriptive sentences that provide a richer semantic signal for the neural model.

2.2 Document Ranker

The expanded query ($Q_e$) is processed by a cross-encoder ranker (VanillaBERT). This model calculates a relevance score by jointly encoding the expanded query and candidate documents. The inclusion of formal definitions aligns the query more closely with the linguistic structures BERT was pre-trained on, compared to simple synonym-based expansion.

The SIR architecture, featuring (A) the Query Expander using ARES and BabelNet, and (B) the Transformer-based Document Ranker.

3. Experimental Evaluation

The model was trained exclusively on the English TREC Robust 2004 dataset. Evaluation was conducted on the same dataset for English performance and on CLEF benchmarks (2000-2008) for zero-shot multilingual performance in French, German, Italian, and Spanish. BabelNet 4.0 served as the underlying knowledge base.

4. Results and Discussion

SIR demonstrated consistent improvements over standard VanillaBERT and BM25+RM3 baselines. In English (Robust04), it achieved a 2.4% improvement in Mean Average Precision (MAP). The gains were more pronounced in multilingual settings; for instance, in Italian benchmarks, SIR improved Precision@20 by approximately 4% and MAP by 10%.

A key insight is that concepts (senses) act as a universal language. By mapping an English query to a BabelNet sense, the system can retrieve definitions in any supported language, facilitating seamless zero-shot transfer. Additionally, the neural ranker proved robust to occasional noise from incorrect sense disambiguation, still benefiting from the added context of correct definitions.

5. Conclusion

SIR effectively bridges the context and language gaps in neural IR. Future work includes improving WSD accuracy beyond nearest-neighbor approaches and optimizing query length by selectively expanding only the most ambiguous terms.

Summary updated

BibTeX
@inproceedings{blloshmi2021ir,
  author = {Rexhina Blloshmi and Tommaso Pasini and Niccolò Campolungo and Somnath Banerjee and Roberto Navigli and Gabriella Pasi},
  title = {{IR like a SIR: Sense-enhanced Information Retrieval for Multiple Languages}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2021},
  doi = {10.18653/v1/2021.emnlp-main.79},
}
Figure