LoL: A Comparative Regularization Loss over Query Reformulation Losses for Pseudo-Relevance Feedback

Yunchang Zhu, Liang Pang, Yanyan Lan, Huawei Shen, Xueqi Cheng

Abstract

Pseudo-Relevance Feedback (PRF) is a cornerstone technique in Information Retrieval (IR) designed to refine user queries by incorporating information from top-retrieved documents. However, PRF often suffers from "query drift," where irrelevant content in the feedback set misleads the model. This paper introduces the Loss-over-Loss (LoL) framework, which applies a comparative regularization during training. By penalizing the model if additional feedback documents degrade performance, LoL enforces the principle that more feedback should yield a better query. Experimental results on MS MARCO and TREC DL benchmarks demonstrate that LoL significantly improves retrieval accuracy and robustness for both sparse and dense neural retrieval models.

Introduction

In modern search systems, users frequently provide short or ambiguous queries that fail to capture their full intent. Pseudo-Relevance Feedback (PRF) addresses this by performing an initial retrieval, assuming the top-$k$ results are relevant, and using them to expand the original query. While effective, the primary challenge of PRF is query drift. Because initial retrieval is imperfect, increasing the number of feedback documents ($k$) often introduces noise. For example, a search for the "Omicron variant" might drift toward ancient Greek history if the feedback set includes documents about the Greek alphabet.

Existing PRF models typically optimize queries based on a fixed $k$ and fail to compare the relative effects of different feedback set sizes. They do not explicitly learn the fundamental intuition of ideal feedback: a robust model should filter out noise such that more feedback documents strictly improve (or at least do not worsen) the reformulated query. This paper proposes the LoL framework to bridge this gap through comparative regularization.

The Loss-over-Loss (LoL) Framework

To teach retrieval models to suppress irrelevant information, the LoL framework introduces a training objective based on the comparison of multiple feedback scenarios.

1. Parallel Revisions

During the training phase, the model generates multiple parallel revisions of a single query using different feedback set sizes (e.g., $k=1, 3, 5$). This allows the model to observe how the inclusion of more documents affects the query representation.

2. Comparative Regularization

The core innovation is the comparative regularization loss, implemented as a pairwise hinge loss. The model calculates a standard ranking loss for each query revision. It then applies a penalty if a revision using more feedback documents ($Query_{k+1}$) results in a higher loss than a revision using fewer documents ($Query_k$). This formalizes the "Comparison Principle":

$$Loss(Query_{k+1}) \le Loss(Query_k)$$

By enforcing this constraint, the neural network is forced to learn how to distinguish between helpful expansion terms and harmful noise, effectively acting as a self-correcting mechanism.

3. Differentiable PRF in Vector Space

To ensure the framework is end-to-end trainable, the authors perform query refinement directly in the embedding space using a BERT-style encoder. By avoiding the non-differentiable step of generating discrete text tokens, the method remains fully differentiable. This flexibility allows LoL to be applied to both dense retrievers (e.g., ANCE), which use semantic embeddings, and sparse retrievers (e.g., uniCOIL), which use term-based importance weights.

Experimental Evaluation

The LoL framework was evaluated using the MS MARCO passage collection (8.8 million passages). Testing was conducted on MS MARCO Dev, TREC DL 2019, TREC DL 2020, and DL-HARD—a benchmark specifically designed for complex queries where PRF is most critical.

Key Results

  • Performance Gains: LoL consistently outperformed base models and existing PRF state-of-the-art methods. On the MS MARCO Dev set, applying LoL to the ANCE model with 5 feedback documents achieved an NDCG@10 of 41.01, a significant jump from the base ANCE score of 38.76 and the previous best (ANCE-PRF) at 40.10.
  • Mitigating Query Drift: A key finding was the stability of LoL as $k$ increased. While baseline PRF models often saw performance peak and then degrade as more documents were added, LoL's performance remained stable or increased monotonically from $k=1$ to $k=5$.
  • Model Agnosticism: The framework proved effective for both sparse and dense architectures. However, dense retrieval showed slightly higher robustness, likely because dense vectors can represent fine-grained semantic nuances better than term-grained sparse vectors, which are more susceptible to polysemy.

Discussion and Limitations

While highly effective, the authors noted that the comparative regularization loss is a "hard constraint" to learn; the loss decreases slowly during training, suggesting that forcing a model to guarantee performance improvements with more data is a complex optimization task. Additionally, while LoL improves sparse retrieval, these models still exhibit a slight vulnerability to query drift compared to their dense counterparts.

Conclusion

The LoL framework introduces a novel way to regularize Pseudo-Relevance Feedback by focusing on the logical relationship between different feedback set sizes. By penalizing query drift through a "Loss-over-Loss" approach, the model learns to effectively ignore noise in the feedback loop. Future work may explore applying this "neglected normal order" regularization to other machine learning tasks where increasing context or data should theoretically yield better predictions.

Summary updated

BibTeX
@inproceedings{zhu2022lol,
  author = {Yunchang Zhu and Liang Pang and Yanyan Lan and Huawei Shen and Xueqi Cheng},
  title = {{LoL: A Comparative Regularization Loss over Query Reformulation Losses for Pseudo-Relevance Feedback}},
  booktitle = {International ACM SIGIR Conference on Research and Development in Information Retrieval},
  year = {2022},
  doi = {10.1145/3477495.3532017},
}
Figure