Query Expansion for IR using Knowledge-Based Relatedness

Arantxa Otegi, Xabier Arregi, Eneko AgirreIJCNLP 2011 · OpenReview

Abstract

Arantxa Otegi, Xabier Arregi, Eneko Agirre. Proceedings of 5th International Joint Conference on Natural Language Processing.

2011.

Introduction: Bridging the Lexical Gap

In Information Retrieval (IR), systems often struggle with the lexical gap—the discrepancy between the vocabulary used in a query and the vocabulary in relevant documents. For instance, a user searching for "automobile" might miss a highly relevant document that only uses the term "car." Traditional keyword-matching systems are inherently limited by this lack of semantic awareness.

To address this, researchers employ Query Expansion (QE). The standard approach, Pseudo-Relevance Feedback (PRF), expands queries using frequent terms from the top-ranked documents of an initial search. While often effective, PRF is prone to topic drift: if the initial results are noisy or slightly off-topic, the expanded query can veer further away from the user's intent, leading to a degradation in performance.

Methods: Relatedness-based Query Expansion (RQE)

This paper proposes Relatedness-based Query Expansion (RQE), a "global" expansion method that leverages external knowledge rather than local document co-occurrence. By using WordNet as a semantic backbone, RQE identifies related concepts independently of the retrieved document set.

The RQE workflow consists of several key steps:

  1. Graph-Based Relatedness: WordNet is treated as a graph of semantic relations. A random walk algorithm (UKB) is initiated from the query terms to determine the most "central" concepts in the graph relative to the query.
  2. Concept Selection: The top $N$ highest-scoring concepts (synsets) are selected.
  3. Term Extraction: All words associated with these top concepts are extracted to form the expansion set.
  4. Integration: The method is integrated into a language modeling framework where the final document score is a weighted combination (parameter $w$) of the original query and the expanded terms.
  5. Smoothing: Dirichlet smoothing ($\mu$) is applied to handle term sparsity, ensuring robust probability estimates.

Datasets

The authors evaluated RQE across three diverse datasets to test its generalizability, ranging from long-form news to short, informal Q&A and formal legal paragraphs.

Dataset statistics showing the variation in document count and average length across the three test collections.
  • Robust-WSD (CLEF 2009): Large-scale news articles with relatively long document lengths.
  • Yahoo! Answers: Short, informal user-generated questions and answers.
  • ResPubliQA: Formal, domain-specific paragraph retrieval from European Union laws.

Key Results and Insights

The experimental results demonstrate that RQE consistently outperforms the standard Query Likelihood (QL) baseline and offers distinct advantages over PRF in specific contexts.

Experimental results comparing QL, PRF, and RQE. Bold values indicate the best performance for each metric.

The "Short Text" Advantage

RQE proved particularly effective for the Yahoo! Answers and ResPubliQA datasets. As shown in the dataset statistics, these collections feature shorter documents where traditional co-occurrence statistics used by PRF may be less reliable. In ResPubliQA, PRF actually degraded performance compared to the baseline (a 5% drop in MRR), whereas RQE provided a significant boost.

Global vs. Local Knowledge

On the Robust-WSD dataset, PRF outperformed RQE (36.69 vs 33.67 MAP). This suggests that for large, diverse news corpora, "local" knowledge derived from the specific document collection is highly effective. However, the authors found that RQE and PRF are complementary. A preliminary combination of both methods on the Robust set achieved a MAP of 37.67, surpassing either method alone. This indicates that semantic relatedness (global) and document co-occurrence (local) capture different, additive dimensions of relevance.

Limitations and Future Work

While promising, RQE introduces some challenges:

  • Parameter Sensitivity: Optimal performance requires tuning parameters like the number of concepts ($N$) and the expansion weight ($w$), which vary by dataset.
  • Knowledge Base Scope: WordNet, while broad, may lack specialized domain terminology. Future work could integrate more specialized knowledge bases like UMLS for medical IR.
  • Computational Cost: Graph-based random walks are more intensive than simple indexing, though optimized tools can mitigate this overhead.

Summary updated

BibTeX
@inproceedings{otegi2011query,
  author = {Arantxa Otegi and Xabier Arregi and Eneko Agirre},
  title = {{Query Expansion for IR using Knowledge-Based Relatedness}},
  booktitle = {IJCNLP},
  year = {2011},
  url = {https://openreview.net/forum?id=Nk33G-0MlDQ},
}
Figure