In modern information retrieval, user clicks are the most abundant source of training data. However, using raw clicks as a proxy for relevance is dangerous. Users do not scan a list of results uniformly; they exhibit a profound position bias, where items at the top receive disproportionately more attention regardless of their actual utility.
If a machine learning model is trained on these raw logs without correction, it enters a self-reinforcing feedback loop. The model learns that top-ranked items are "good" simply because they were clicked, and it continues to rank them highly, effectively burying potentially superior content that never had the chance to be seen.
The Geometry of Position Bias
The core challenge, as identified in the seminal work Accurately interpreting clickthrough data as implicit feedback by Thorsten Joachims and colleagues, is that a click is a confounded signal. A click on an item at rank $k$ is the product of two distinct probabilities: the probability that the user examined the item and the probability that the item is relevant.
Hila Becker, Christopher Meek, and David Maxwell Chickering demonstrated in Modeling Contextual Factors of Click Rates that position is the primary external factor influencing engagement. This creates a "trust bias," where users click the first result not because it is the best, but because they trust the system's ordering. To break this cycle, researchers like Filip Radlinski proposed Minimally Invasive Randomization, which swaps items to observe how behavior changes when high-quality content is moved to lower positions.
Counterfactual Learning and Propensity Scoring
The breakthrough in training models on biased data without constant randomization came from Counterfactual Learning-to-Rank (ULTR). In Unbiased Learning-to-Rank with Biased Feedback, Thorsten Joachims, Adith Swaminathan, and Tobias Schnabel introduced the use of Inverse Propensity Scoring (IPS).
The intuition is simple: if an item at rank 10 is only seen 10% as often as an item at rank 1, a click at rank 10 should "count" ten times as much during training. By re-weighting clicks by the inverse of their examination probability (the propensity), the algorithm can theoretically recover an unbiased ranking model as if it were trained on perfect, expert-labeled data. Aman Agarwal and others later generalized this in Counterfactual Learning-to-Rank for Additive Metrics and Deep Models, allowing these corrections to work with complex neural networks and metrics like DCG.
The Confounding of Relevance and Rank
A significant hurdle arises when the system used to collect data is already quite good. In such cases, high-quality items are almost always at the top. This creates a correlation between relevance and position that makes it difficult for models to distinguish between the two.
In Towards Disentangling Relevance and Bias in Unbiased Learning to Rank, Yunan Zhang and Zhen Qin identified that in "two-tower" architectures—where one tower models relevance and another models bias—the bias tower often "steals" relevance information. If the model sees that position 1 always gets clicks, it might simply learn to predict "high relevance" for anything at position 1, failing to learn the actual features of the content. To combat this, techniques like Gradient Reversal and Observation Dropout are used to force the towers to truly disentangle the two signals.
Similarly, Yao Zhao and Zhining Liu argued in Mitigate Position Bias with Coupled Ranking Bias on CTR Prediction that standard debiasing often overestimates position bias because it ignores the fact that the system intentionally put better items at the top. Their Gradient Interpolation method provides a way to balance these two effects.
Societal and Systemic Implications
Unbiased learning is not just about accuracy; it is about fairness and discovery. Tao Yang and Chen Luo noted in Mitigating Exploitation Bias in Learning to Rank that behavioral signals often drown out content features, preventing new or "cold-start" items from ever being discovered.
This "rich-get-richer" effect has real-world consequences. Mohit Singhal and Javier F. Pacheco showed in Auditing Yelp's Business Ranking how these feedback loops can favor established businesses in wealthy areas. Even in high-stakes professional fields, Beining Wang and Qingyao Ai found that the ranking of legal cases acts as a psychological anchor, steering judicial decisions toward the top-ranked results (Investigating the Influence of Legal Case Retrieval Systems).
By treating ranking as a causal inference problem rather than a simple supervised learning task, the field of ULTR seeks to build systems that learn from human behavior without being enslaved by its biases.