Stabilizing Inverse Propensity Weighting for Causal Inference

368 concepts247 papers2 proposals

When estimating causal effects from observational data, we must account for confounding—the fact that certain types of individuals are more likely to receive a treatment than others. A foundational tool for this task is the propensity score, defined as the probability that a participant receives a treatment given their observed characteristics.

In an ideal world, we use these scores to re-weight our data via Inverse Propensity Weighting (IPW). By weighting each individual by the inverse of their probability of receiving their actual treatment, we synthetically create a "pseudo-population" where the treatment and control groups are perfectly balanced.

However, this elegant mathematical theory frequently collides with a harsh numerical reality: propensity score instability. When the probability of treatment assignment approaches the boundaries of zero or one, the inverse of that probability explodes, causing statistical estimators to break down.

The Intuition Behind the Breakdown

To understand why IPW fails under extreme propensity scores, imagine a clinical study evaluating a new medication. Suppose a specific subpopulation—for instance, patients over the age of 80 with a history of heart disease—is almost never prescribed this medication due to safety concerns. Out of thousands of patients in the data, perhaps only one 81-year-old with heart disease receives the treatment, giving them a propensity score of $e(X) = 0.001$.

When we apply IPW, this single individual receives a weight of:

$$\frac{1}{0.001} = 1000$$

In our synthetic pseudo-population, this single patient now represents 1,000 people. If this patient happens to have an unusually positive or negative outcome purely by chance, their single data point will violently swing the entire average treatment effect estimate.

As noted by Zenati et al., this lack of covariate overlap between treatment groups leads to estimators that suffer from extreme variance and are overly sensitive to individual data points. When the "positivity assumption"—the requirement that every individual has a non-zero probability of receiving every treatment—is violated or nearly violated, the denominator in our estimator vanishes, causing variance to skyrocket.

This instability is not unique to simple average treatment effects. Xu et al. demonstrate that standard causal estimators become highly inaccurate when faced with extreme propensity scores across both back-door and front-door adjustment frameworks. Furthermore, Ma et al. show that exploding weights prevent the estimator from following a standard normal distribution, which invalidates traditional confidence intervals and hypothesis testing.

Where Instability Hurts Most

Propensity score instability propagates through many advanced machine learning and statistical domains:

  • Survival Analysis: In longitudinal studies with right-censored data, researchers must model both the treatment assignment and the probability of remaining uncensored. As Wang et al. point out, dividing by extreme propensity scores in survival models causes variance spikes that make long-term risk estimation highly unreliable.
  • Recommendation Systems: Recommender systems can be framed causally, where showing an item is a treatment. However, Bonner notes that because the vast majority of items are rarely recommended, the logged interaction data is incredibly sparse, causing traditional IPW evaluation metrics to exhibit massive variance.
  • Mediation Analysis: When decomposing an effect into direct and indirect pathways, the estimator must account for the joint probabilities of both the treatment and the intermediate mediator. Miles et al. show that rare combinations of treatments and mediators create severe positivity violations, undermining numerical reliability.

Strategies for Stabilization

To combat this variance explosion, researchers have developed several distinct strategies that either smooth the extreme weights, change how the data is matched, or bypass the propensity score denominator entirely.

1. Weight Trimming and Tail Smoothing

The simplest historical fix to extreme weights is trimming—manually capping weights at a maximum threshold or discarding individuals with propensity scores outside a certain window (e.g., $[0.05, 0.95]$). However, as Liu et al. argue, hard trimming introduces severe bias and makes the final estimates highly sensitive to the arbitrarily chosen threshold.

To solve this, Zhu et al. introduced a data-driven alternative called Pareto-smoothed inverse propensity weighting. Instead of abruptly cutting off large weights, the authors model the upper tail of the weight distribution using a generalized Pareto distribution. This smoothly compresses the extreme outliers back toward a manageable range, drastically reducing variance in small sample sizes without introducing the harsh bias of hard truncation.

2. Matching with Replacement

An alternative to weighting is propensity score matching, where treated units are paired with control units that have similar propensity scores. Wang et al. demonstrate that when covariate overlap is weak, matching with replacement provides significantly higher numerical stability than IPW. By reusing actual patient outcomes rather than inflating sparse data points with massive weights, matching avoids the denominator-driven variance spikes that plague weighted estimators.

3. Bypassing the Denominator via Meta-Learners

Another powerful paradigm shifts the estimation burden away from the propensity model and onto the outcome model. Causal meta-learners use underlying machine learning algorithms to predict potential outcomes directly.

Acharki et al. show that meta-learners like the X-learner and DR-learner (Doubly Robust learner) are inherently more robust to poor overlap than pure IPW methods. Because these cross-procedures rely primarily on modeling the response surfaces rather than inverting assignment probabilities, they maintain stability even when certain treatment levels are rarely observed for specific subgroups.

4. Representation Learning and Balancing Weights

In modern deep learning approaches to causal inference, researchers attempt to learn a latent representation space where the treatment and control groups look identical. However, forcing strict balance can inadvertently erase features that are highly predictive of the outcome.

To address this, Assaad et al. propose Balancing Weights Counterfactual Regression (BWCFR). Instead of using standard IPW, they incorporate targeted balancing weights directly into the representation learning objective. This focuses the model's attention on regions of the covariate space where substantial overlap actually exists, preventing the network from making erratic counterfactual predictions in sparse data zones.

Similarly, Bahadori et al. introduce End-to-End Balancing (E2B) for continuous treatments. Rather than estimating propensity scores as an intermediate step, E2B directly optimizes the base weights of an entropy-balancing objective to minimize the error in the final causal response curve, preventing individual data points from exerting disproportionate influence over the results.

History & citationVersion 2 · 2026-10-04

Current · Permanent link

  1. Version 2 (latest) · 2026-10-04 · 252 papers
  2. Version 1 · 2026-10-04 · 283 papers
Cite this direction

Latest version (a living page):

@misc{lacuna_propensity_score_instability_153e2162,
  title = {propensity score instability},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/propensity-score-instability/rdr_153e2162489643fa892c0a7240541360},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 2, frozen text):

@misc{lacuna_propensity_score_instability_153e2162_v2,
  title = {propensity score instability},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/propensity-score-instability/rdr_153e2162489643fa892c0a7240541360/v/2},
  note = {Version 2, frozen 2026-10-04. Accessed 2026-11-20},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake