When you search for something online, you are highly likely to click one of the top three results. Is it because those results are perfectly tailored to your query, or simply because they are at the top?
This is the core dilemma of modern information retrieval. Modern search and recommendation engines rely heavily on implicit user feedback—such as clicks—to train their machine learning models. Clicks are incredibly abundant and cheap compared to hiring human experts to manually score documents. However, as Becker et al. demonstrate, an item's physical position on a page is often the primary factor driving its click-through rate, completely independent of its actual quality.
This phenomenon is known as position bias (or presentation bias). If a ranking system trains directly on raw click logs without adjusting for this bias, it creates a destructive feedback loop: the system places an item at the top because it thinks it is relevant, the item receives clicks merely because it is at the top, and the model interprets those clicks as confirmation that its choice was correct (Raman et al.; Gao et al.).
Unbiased Learning to Rank (ULTR) is a framework designed to break this loop. It allows us to mathematically disentangle user preference from presentation artifacts, extracting clean relevance signals from fundamentally biased human behavior.
Decomposing the Click Signal
To understand why ULTR works, we must first model how a user interacts with a list of items. The foundational intuition relies on the Position-Based Model (PBM), which factorizes the probability of a user clicking an item into two independent, latent variables:
$$\text{P}(\text{Click}) = \text{P}(\text{Examination}) \times \text{P}(\text{Relevance})$$
- $\text{P}(\text{Examination})$: The probability that the user actually looks at the item. This is dictated almost entirely by the item's position on the screen.
- $\text{P}(\text{Relevance})$: The probability that the user finds the item useful given that they have looked at it. This is the intrinsic utility we want our search engine to learn.
In a standard click log, we only observe the binary outcome of $\text{P}(\text{Click})$. If an item at position 10 receives no clicks, we do not know if it was rejected because it was irrelevant, or if the user simply stopped scrolling at position 3 and never saw it. As Junpei Komiyama notes, this creates an identifiability problem where item quality and positional visibility are deeply confounded.
Early attempts to isolate these variables relied on behavioral heuristics or explicit system manipulation. For instance, Joachims et al. utilized eye-tracking experiments and rank-reversal perturbations to show that users possess an inherent "trust bias" toward top results. They introduced the click-skip heuristic: if a user clicks an item at position 3, they must have scanned positions 1 and 2. Therefore, if positions 1 and 2 were skipped, the item at position 3 can be inferred to be more relevant than the skipped items above it.
While heuristics help filter out some bias, they do not scale well to complex user interfaces and discard a massive amount of training data.
Counterfactual Estimation via Inverse Propensity Scoring
The breakthrough of modern ULTR is treating position bias as a causal inference problem. If we view the display position as a "treatment" and the examination probability as a "propensity score," we can apply Inverse Propensity Scoring (IPS) to correct the data distribution counterfactually.
Imagine we know from historical exploration data that users only look at position 10 about 10% of the time ($\text{P}(\text{Examination} \mid \text{position } 10) = 0.1$). If an item manages to get a click at position 10, that click is a massive signal of relevance. To correct for the fact that 90% of users missed this item entirely, the IPS framework weights that single click by the inverse of its examination probability:
$$\text{Weight} = \frac{1}{\text{P}(\text{Examination})} = \frac{1}{0.1} = 10$$
By counting that click as 10 clicks during model training, we mathematically reconstruct an unbiased training distribution—simulating what user engagement would look like if every single item had been displayed in the top position.
To implement this in production, systems must accurately estimate these propensity scores. Langford et al. pioneered "exploration scavenging," showing that you can isolate position bias by comparing the performance of identical items when they happen to appear at different ranks due to natural algorithmic variance. More recently, Vinay et al. introduced parametric imitation rankers to estimate these propensities offline, allowing developers to evaluate new ranking algorithms safely without deploying risky, randomized A/B tests to live users.
The Dual Learning Architecture
While early IPS methods required a two-step process—first running randomized traffic to estimate propensities, then training the ranker—modern industrial frameworks like TF-Ranking (Pasumarthi et al.) often use a Dual Learning Algorithm (DLA).
DLA sets up a cooperative game between two neural networks trained simultaneously on the same click logs:
- The Relevance Model takes item features (text, price, category) as input and tries to predict true relevance, using the current position model to debias its loss function.
- The Position Model takes the display rank as input and tries to isolate the examination curve, using the current relevance model to factor out the intrinsic quality of the items that happened to be shown at those ranks.
By optimizing both networks jointly, the system learns to separate the presentation layout from the underlying content utility without requiring continuous, user-annoying randomization.
The Reality Gap in Production
While the mathematical foundations of counterfactual learning are elegant, deploying these systems at scale reveals a significant gap between simulation and reality.
Hager et al. evaluated state-of-the-art ULTR models on Baidu's massive real-world search dataset. They discovered that while debiasing algorithms significantly improve a model's ability to predict future clicks, these gains do not always translate to better rankings when evaluated against expert-annotated relevance.
Why does this happen? Real-world user behavior is far more intricate than the simple independence assumptions of the Position-Based Model. For example:
- Snippet Quality: A highly relevant document might have a poorly generated text snippet on the search results page, causing users to skip it even if they look directly at it.
- The Rich-Get-Richer Problem: When a production ranker is already highly optimized, relevant documents are heavily concentrated at the top. As Niu et al. point out, this leaves lower positions starved of high-quality content. Without sufficient exploration data at lower ranks, counterfactual models suffer from high variance and struggle to calculate stable propensity scores.
Ultimately, mitigating position bias is not just about applying a static mathematical correction. It requires an ongoing balance between exploiting what the system already knows to satisfy users today, and strategically injecting subtle perturbations (Erick Delage) to discover the hidden relevance of tomorrow.