Abstract
Arantxa Otegi, Xabier Arregi, Eneko Agirre. Proceedings of 5th International Joint Conference on Natural Language Processing.
2011.
Introduction: Bridging the Lexical Gap
In Information Retrieval (IR), systems often struggle with the lexical gap—the discrepancy between the vocabulary used in a query and the vocabulary in relevant documents. For instance, a user searching for "automobile" might miss a highly relevant document that only uses the term "car." Traditional keyword-matching systems are inherently limited by this lack of semantic awareness.
To address this, researchers employ Query Expansion (QE). The standard approach, Pseudo-Relevance Feedback (PRF), expands queries using frequent terms from the top-ranked documents of an initial search. While often effective, PRF is prone to topic drift: if the initial results are noisy or slightly off-topic, the expanded query can veer further away from the user's intent, leading to a degradation in performance.
Methods: Relatedness-based Query Expansion (RQE)
This paper proposes Relatedness-based Query Expansion (RQE), a "global" expansion method that leverages external knowledge rather than local document co-occurrence. By using WordNet as a semantic backbone, RQE identifies related concepts independently of the retrieved document set.
The RQE workflow consists of several key steps:
- Graph-Based Relatedness: WordNet is treated as a graph of semantic relations. A random walk algorithm (UKB) is initiated from the query terms to determine the most "central" concepts in the graph relative to the query.
- Concept Selection: The top $N$ highest-scoring concepts (synsets) are selected.
- Term Extraction: All words associated with these top concepts are extracted to form the expansion set.
- Integration: The method is integrated into a language modeling framework where the final document score is a weighted combination (parameter $w$) of the original query and the expanded terms.
- Smoothing: Dirichlet smoothing ($\mu$) is applied to handle term sparsity, ensuring robust probability estimates.
Datasets
The authors evaluated RQE across three diverse datasets to test its generalizability, ranging from long-form news to short, informal Q&A and formal legal paragraphs.
- Robust-WSD (CLEF 2009): Large-scale news articles with relatively long document lengths.
- Yahoo! Answers: Short, informal user-generated questions and answers.
- ResPubliQA: Formal, domain-specific paragraph retrieval from European Union laws.
Key Results and Insights
The experimental results demonstrate that RQE consistently outperforms the standard Query Likelihood (QL) baseline and offers distinct advantages over PRF in specific contexts.
The "Short Text" Advantage
RQE proved particularly effective for the Yahoo! Answers and ResPubliQA datasets. As shown in the dataset statistics, these collections feature shorter documents where traditional co-occurrence statistics used by PRF may be less reliable. In ResPubliQA, PRF actually degraded performance compared to the baseline (a 5% drop in MRR), whereas RQE provided a significant boost.
Global vs. Local Knowledge
On the Robust-WSD dataset, PRF outperformed RQE (36.69 vs 33.67 MAP). This suggests that for large, diverse news corpora, "local" knowledge derived from the specific document collection is highly effective. However, the authors found that RQE and PRF are complementary. A preliminary combination of both methods on the Robust set achieved a MAP of 37.67, surpassing either method alone. This indicates that semantic relatedness (global) and document co-occurrence (local) capture different, additive dimensions of relevance.
Limitations and Future Work
While promising, RQE introduces some challenges:
- Parameter Sensitivity: Optimal performance requires tuning parameters like the number of concepts ($N$) and the expansion weight ($w$), which vary by dataset.
- Knowledge Base Scope: WordNet, while broad, may lack specialized domain terminology. Future work could integrate more specialized knowledge bases like UMLS for medical IR.
- Computational Cost: Graph-based random walks are more intensive than simple indexing, though optimized tools can mitigate this overhead.
Summary updated