Recent Advances in the Foundations and Applications of Unbiased Learning to Rank

Shashank Gupta, Philipp Hager, Jin Huang, Ali Vardasbi, Harrie Oosterhuis

This paper is a summary of a tutorial presented at the SIGIR '23 conference. It provides a high-level roadmap of the recent evolution in Unbiased Learning to Rank (ULTR), moving from foundational theories to complex, real-world applications.

TL;DR

This paper outlines the state-of-the-art in Unbiased Learning to Rank (ULTR), a field dedicated to training high-quality search and recommendation models using "noisy," biased user click data rather than expensive manual labels. It highlights the shift from simple position-bias correction to sophisticated causal inference and "doubly robust" methods that handle complex user behaviors and fairness requirements.

What Problem Does This Paper Address?

Modern search engines and recommendation systems rely on Learning to Rank (LTR). Traditionally, these models were trained on manual relevance judgments—human experts grading how "good" a result is. This is slow, expensive, and often fails to capture what actual users want.

The alternative is using click data, which is abundant and free. However, clicks are "biased." A user might click the first result simply because it’s at the top (position bias) or because they trust the brand (trust bias), even if the third result is more relevant. If you train a model on this raw data, it will simply learn to replicate these biases. ULTR aims to mathematically "debias" this interaction data to uncover the true underlying relevance of items.

Methods

The paper categorizes the latest techniques used to solve the bias problem, moving beyond the traditional Inverse Propensity Scoring (IPS)—a method that weights clicks based on the probability of an item being seen.

  • Advanced Estimation: It introduces Doubly Robust Estimation, which combines IPS with a predictive model of user behavior to reduce the high variance (instability) often found in pure IPS.
  • Affine Corrections: A newer technique designed to handle "trust bias," where IPS typically fails.
  • Causal Inference: The field is moving toward causal-inference-based methods (like Heckerman’s method) and Two-Tower Models, which use separate neural network branches to model "relevance" and "bias" independently.
  • Online & Counterfactual Integration: Traditionally, ULTR was "offline" (using old logs). New methods aim to unify this with "online" LTR, where the model learns and debiases in real-time as users interact with the system.
  • Safe Policy Learning: Techniques that use "risk regularization" to ensure that a newly learned ranking model isn't significantly worse than the current production system before it is deployed.

Datasets

As this is a tutorial summary and survey of the field, it does not introduce a new dataset or perform a single experiment. Instead, it references a wide range of real-world application areas where these methods have been tested, including:

  • Top-K Ranking: Standard search result lists.
  • Product Feeds: 2D grid-based interfaces (like Amazon or Instagram).
  • Job Recommendations: Systems where "recency bias" (preferring new postings) is a major factor.

Key Insights and Ideas

  • IPS is not a Silver Bullet: While IPS was the "gold standard" for years, the authors highlight that it is ineffective against certain biases like trust bias and can suffer from extreme mathematical instability (high variance) when data is sparse.
  • The Convergence of Bias and Fairness: One of the most interesting "aha moments" is the link between ULTR and Fairness in Ranking. If a system is biased against certain items due to their position, it is inherently "unfair" to the creators of those items. The mathematical tools used to debias clicks are now being used to ensure "Equity of Attention" for different groups.
  • From Formulas to Architectures: Early ULTR was mostly about clever mathematical weighting formulas. Recent trends show a shift toward architectural solutions, such as two-tower neural networks, which bake the understanding of bias directly into the model's structure.

Key Results

The paper summarizes the impact of these advances across the industry:

  • Improved Robustness: Doubly robust methods have shown significant performance gains over standard IPS in offline bandit learning settings.
  • Handling Complex UI: ULTR has successfully moved from simple 1D lists to complex 2D "feeds," accounting for how users scan grids versus vertical lists.
  • Deployment Safety: The introduction of "Safe Counterfactual Policy Learning" provides theoretical guarantees that allow companies to deploy ULTR models without risking a sudden drop in user satisfaction.

Limitations and Future Work

  • The "End of Unbiasedness": The authors acknowledge recent research suggesting there are fundamental, implicit limitations to how much we can learn from clicks alone. Some biases may be so deep that they cannot be fully "undone" mathematically.
  • Variance Issues: Despite the move to doubly robust methods, high variance remains a challenge when dealing with "long-tail" queries where very little click data exists.
  • Future Directions: The authors suggest that the next frontier is the further unification of online and offline learning, and deeper integration of ULTR with multi-stakeholder fairness (balancing the needs of users, platforms, and content creators).

Summary updated

Abstract

Since its inception, the field of unbiased learning to rank (ULTR) has remained very active and has seen several impactful advancements in recent years. This tutorial provides both an introduction to the core concepts of the field and an overview of recent advancements in its foundations along with several applications of its methods. The tutorial is divided into four parts: Firstly, we give an overview of the different forms of bias that can be addressed with ULTR methods. Secondly, we present a comprehensive discussion of the latest estimation techniques in the ULTR field. Thirdly, we survey published results of ULTR in real-world applications. Fourthly, we discuss the connection between ULTR and fairness in ranking. We end by briefly reflecting on the future of ULTR research and its applications. This tutorial is intended to benefit both researchers and industry practitioners who are interested in developing new ULTR solutions or utilizing them in real-world applications.
BibTeX
@inproceedings{gupta2023recent,
  author = {Shashank Gupta and Philipp Hager and Jin Huang and Ali Vardasbi and Harrie Oosterhuis},
  title = {{Recent Advances in the Foundations and Applications of Unbiased Learning to Rank}},
  booktitle = {International ACM SIGIR Conference on Research and Development in Information Retrieval},
  year = {2023},
  doi = {10.1145/3539618.3594247},
}

Loading related content...

Figure