Structure Cognizant Pseudo Relevance Feedback

Arjun Atreya V, Yogesh Kakde, Pushpak Bhattacharyya, Ganesh RamakrishnanIJCNLP 2013 · OpenReview

Abstract

Traditional Pseudo Relevance Feedback (PRF) techniques often treat documents as flat "bags of words," neglecting the rich semantic information embedded in document structure. This paper presents StructPRF, a novel framework developed by researchers at IIT Bombay that incorporates document structure into the query expansion process. By recognizing that terms in specific fields—such as Titles or Infoboxes—are more representative of a document's core topic, StructPRF significantly improves retrieval accuracy. Experimental results across four languages demonstrate performance gains of up to 30% over traditional PRF methods.

Introduction

In Information Retrieval (IR), Pseudo Relevance Feedback (PRF) is a widely used technique to improve search results by expanding the original query with terms from the top-ranked documents. However, standard PRF models typically ignore the internal structure of these documents. In structured repositories like Wikipedia, different fields (Title, Body, Infobox, Categories) carry varying degrees of importance. A term appearing in a document's title is likely a central concept, whereas a term in the body might be a peripheral mention. StructPRF addresses this missed opportunity by weighting expansion terms based on their field of origin.

Methodology

The StructPRF framework integrates document structure into the feedback loop through a multi-step process:

  1. Per-Field EM Formulation: Instead of a single Expectation-Maximization (EM) algorithm for the entire document, StructPRF runs separate EM processes for each field: Title, Body, Category, and Infobox.
  2. Weighted Combination: The final weight of an expansion term ($P_{Rk}$) is calculated as a weighted sum of its importance across these fields: $$P_{Rk} = \alpha_t \cdot P_{Rtk} + \alpha_b \cdot P_{Rbk} + \alpha_c \cdot P_{Rck} + \alpha_i \cdot P_{Rik}$$ Empirical testing determined the optimal hierarchy of importance to be Title ($\alpha = 0.4$) > Body > Categories > Infobox ($\alpha = 0.1$).

  3. Query Aggregation: To prevent "topic drift," where the expanded query deviates from the user's intent, the new terms are merged with the original query using a balancing parameter ($\lambda$).

Experimental Setup

The system was evaluated using standard IR benchmarks across four diverse languages:

  • English & Hindi: FIRE 2010 dataset.
  • Spanish & Finnish: ELRA-E0036 dataset (CLEF). Wikipedia served as the external knowledge base for term expansion due to its consistent structure and comprehensive coverage.

Results and Analysis

The primary metric for evaluation was Mean Average Precision (MAP), comparing StructPRF against a baseline with no feedback (NORF) and traditional "vanilla" PRF.

Precision curves across four languages comparing StructPRF, PRF, and NORF.

Key findings include:

  • Significant Performance Gains: Spanish saw the most dramatic improvement, with StructPRF outperforming vanilla PRF by approximately 30%. English showed an 8% improvement, while Hindi and Finnish saw gains between 2% and 7%.
  • Precision at 'k': As shown in the figures above, StructPRF consistently maintains higher precision in the top-ranked results compared to traditional methods. This is critical for real-world search engines where users rarely look past the first page.
  • Universality: The consistent improvements across four different language families suggest that document structure is a universal proxy for relevance in human-organized information.

Conclusion and Future Work

StructPRF demonstrates that document structure is a powerful signal for improving search relevance. By prioritizing fields like Titles, the system naturally filters for the most representative expansion terms.

Future research directions include:

  • Automated Weighting: Moving beyond manual tuning to automatically learn field weights ($\alpha$ values) for different query types.
  • Tight Coupling: Exploring a single EM algorithm over individual per-field distributions rather than combining separate EM outputs.
  • Domain Adaptation: Testing the model on less structured web content where standard fields like "Infobox" may be absent.

Summary updated

BibTeX
@inproceedings{v2013structure,
  author = {Arjun Atreya V and Yogesh Kakde and Pushpak Bhattacharyya and Ganesh Ramakrishnan},
  title = {{Structure Cognizant Pseudo Relevance Feedback}},
  booktitle = {IJCNLP},
  year = {2013},
  url = {https://openreview.net/forum?id=n5DtkkpI5f},
}
Figure