Counterfactual Debiasing for Recommendation and Ranking Systems

158 concepts108 papers1 proposal

When training search engines and recommender systems, machine learning models are typically fed historical interaction logs—records of what users clicked, watched, or rated. On the surface, this seems logical: a good model should learn to predict what users like.

However, this data contains a fundamental flaw: it is Missing Not At Random (MNAR). A user cannot click on an item they were never shown, and they are far more likely to notice an item displayed at the very top of a page than one buried three screens down. If a model is trained directly on these biased logs, it implicitly learns to replicate the presentation choices of the historical system rather than the true preferences of the users. This creates a feedback loop where popular or highly ranked items continue to receive exposure, while relevant but less visible items are ignored.

To break this loop, a major research direction treats historical logs not as direct ground truth, but as observational data from a confounded experiment. By applying causal inference techniques—specifically Inverse Propensity Weighting (IPW) or Inverse Propensity Scoring (IPS)—we can mathematically simulate an unbiased counterfactual world where every item has an equal opportunity to be seen.


The Core Intuition: Recommendations as Treatments

To understand why propensity weighting works, it helps to reframe the recommendation problem using the language of causal inference. As established by Schnabel et al. (2016), displaying an item to a user can be viewed as an active intervention or a "treatment." The user's subsequent interaction (a click or a rating) is the observed outcome.

In an ideal world, we would evaluate a model using a randomized controlled trial where every user is exposed to every item uniformly. Since this is prohibitively expensive and degrades user experience, we must rely on historical logs where the "treatment assignment" was highly biased.

Inverse Propensity Scoring corrects for this bias by scaling each observed interaction by the reciprocal of its exposure probability (the propensity score).

$$\text{Weighted Loss} = \frac{\text{Observed Loss}}{\mathbb{P}(\text{Exposure})}$$

To build intuition, consider two scenarios:

  1. A click at Position 1: The system's historical policy makes exposure almost certain ($\mathbb{P}(\text{Exposure}) = 1.0$). The inverse weight is $1 / 1.0 = 1$. This interaction is taken at face value.
  2. A click at Position 10: The user rarely scrolls this far ($\mathbb{P}(\text{Exposure}) = 0.10$). If a user actually finds and clicks an item here, it represents an exceptionally strong signal of relevance. The inverse weight scales this loss contribution by $1 / 0.10 = 10$.

By amplifying the value of rare interactions and down-weighting highly predictable ones, the optimization objective aligns with the true underlying user preferences rather than presentation artifacts (Schnabel et al., 2016; Sato et al., 2020).


Estimating the Propensities

The mathematical validity of counterfactual learning hinges on knowing the exact probability of exposure, $\mathbb{P}(\text{Exposure})$. Because these true probabilities are rarely recorded directly, researchers have developed several sophisticated frameworks to estimate them from user behavior.

Position-Based and Cascade Models

In Unbiased Learning to Rank (ULTR), the classical approach assumes a Position-Based Model (PBM). Here, a click is a joint product of two independent factors: whether a user examines a position, and whether the document at that position is relevant (Jagerman et al., 2019; Oosterhuis et al., 2020).

To make this more realistic, researchers have expanded the context of these models:

  • Contextual Variation: Agarwal et al. (2018) introduced the Contextual Position-Based Model (CPBM), which estimates examination probabilities by harvesting natural variations across different user and query contexts from historical A/B test logs.
  • Sequential Dependencies: Real-world browsing is often sequential rather than independent. Vardasbi et al. (2020) proposed a cascade-based propensity estimator (CM-IPS) that models session-specific dependencies, recognizing that the probability of looking at a result depends heavily on the relevance of the results above it.
  • Device-Specific Behavior: In mobile environments where screen real estate alters scrolling habits, Niu et al. (2022) demonstrated that leveraging a specialized Mobile Click Model (MCM) provides significantly more accurate propensity weights than generic position models.

Cross-Dataset Estimation

When explicit feedback (like star ratings) is highly biased because users only rate items they enjoy, Li et al. (2022) demonstrated that a secondary, less-biased dataset of implicit feedback (like raw clicks) can be used to estimate the propensity scores needed to debias the high-information rating dataset.


Beyond Multiplicative Weights: Trust Bias and Affine Corrections

Standard IPS relies on a strict multiplicative assumption: if an item is not examined, it cannot be clicked. However, real-world user behavior violates this assumption through trust bias—the tendency for users to click on top-ranked items simply because they trust the search engine, even if the snippet is irrelevant.

Vardasbi et al. (2020) proved that standard multiplicative IPS is fundamentally incapable of eliminating trust bias. To solve this, the authors introduced an affine estimator. Instead of a simple multiplier, the correction uses an affine transformation that incorporates both a position-based examination weight and a rank-based trust offset:

$$\mathbb{P}(\text{Click}) = \mathbb{P}(\text{Examination}) \times \mathbb{P}(\text{Relevance}) + \text{Trust Offset}$$

By utilizing an Expectation-Maximization framework to isolate the trust offset from historical click logs, this approach allows ranking models to recover an entirely unbiased relevance signal even in the presence of systemic user trust (Vardasbi et al., 2020; Gupta et al., 2023).


Multi-Stage and Cascaded Pipelines

Industrial systems rarely rely on a single model to rank items. Instead, they deploy multi-stage pipelines: a fast, high-recall candidate generator filters millions of items down to a few hundred, and a complex, high-precision re-ranker determines the final presentation order.

This architecture introduces a cumulative selection bias. If an item is dropped by the candidate generator, its exposure probability at the ranking stage is zero. As Gupta et al. (2025) demonstrate, standard single-stage estimators fail in these environments because they ignore the filtering behavior of the earlier stages.

To achieve unbiased learning across the entire system, the authors developed a multi-stage inverse propensity scoring framework. This method calculates the joint probability of an item surviving all preceding selection filters, allowing the final re-ranker to accurately estimate true item utility from logs that have been heavily pre-filtered (Gupta et al., 2025).


The Hidden Connection to Contrastive Learning

While counterfactual learning traditionally requires explicitly modeling and inverting propensity scores, recent theoretical insights have revealed that some popular deep learning objectives perform this process implicitly.

Zhou et al. (2021) established a mathematical bridge between Contrastive Learning and Inverse Propensity Weighting. They proved that when optimizing a contrastive loss function (such as InfoNCE) where negative samples are drawn directly from the data distribution, the objective function becomes mathematically equivalent to an IPW estimator.

Because popular items appear frequently in the dataset, they are sampled as negatives more often, which automatically penalizes their influence. Conversely, rare items receive an implicit boost. This equivalence allows large-scale systems to mitigate exposure bias naturally without the operational complexity of maintaining a separate, explicit propensity model (Zhou et al., 2021).


Practical Frontiers and the "Reality Gap"

Despite its elegant theoretical foundations, deploying counterfactual learning in production presents distinct engineering and statistical challenges.

  • Deterministic Logging Policies: Traditional importance sampling breaks down when a system's logging policy is completely rigid and deterministic, as the propensity for unselected items drops to zero. To bypass this limitation, Tanaka et al. (2026) demonstrated that transitioning from ranking-level weights to click-based importance weights allows for accurate off-policy evaluation even when the underlying logging system allows no stochastic exploration.
  • The Reality Gap: While counterfactual estimators consistently outperform naive baselines in simulated environments, real-world deployment can reveal a disconnect. In a comprehensive study using massive industrial search logs, Hager et al. (2024) found that while propensity weighting significantly improves click prediction metrics, these statistical gains do not always translate cleanly into improvements on expert-annotated relevance metrics.

This ongoing tension highlights the current frontier of the field: refining propensity models to capture the messy, multi-faceted nature of human intent while managing the variance inherent to inverse-weighted statistical estimators.

Key Foundational Work

Recommendations as Treatments: Debiasing Learning and Evaluation (2016) — contributed 3 concept elements to this direction.
History & citationVersion 2 · 2026-10-04

Current · Permanent link

  1. Version 2 (latest) · 2026-10-04 · 108 papers
  2. Version 1 · 2026-10-04 · 70 papers
Cite this direction

Latest version (a living page):

@misc{lacuna_inverse_propensity_weighting_cac29c66,
  title = {inverse propensity weighting},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/inverse-propensity-weighting/rdr_cac29c66a45543ef911eb0d12ea080d2},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-19},
}

This exact version (version 2, frozen text):

@misc{lacuna_inverse_propensity_weighting_cac29c66_v2,
  title = {inverse propensity weighting},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/inverse-propensity-weighting/rdr_cac29c66a45543ef911eb0d12ea080d2/v/2},
  note = {Version 2, frozen 2026-10-04. Accessed 2026-11-19},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake