TL;DR
This paper provides a comprehensive tutorial on propensity score (PS) weighting methods, specifically addressing situations where the "positivity" assumption is violated. It introduces and unifies a suite of alternative estimands (WATE, WATT, WATC) that offer valuable insights into treatment effects by focusing on specific subpopulations, and provides an R package, ChiPS, for practical implementation.
What Problem Does This Paper Address?
In observational studies, researchers often aim to estimate the effect of a treatment or exposure on an outcome. A key assumption for many standard methods (like estimating the Average Treatment Effect - ATE) is "positivity," which essentially means that every individual has a non-zero probability of receiving either the treatment or the control, regardless of their characteristics. When this assumption is violated, standard methods can produce unstable or even unidentifiable results. This can happen structurally (e.g., certain groups are inherently unable to receive a treatment) or randomly (e.g., due to small sample sizes or model misspecification). This paper addresses the challenge of drawing valid causal inferences when positivity is violated by introducing and explaining alternative estimands that target specific subpopulations where positivity holds, thereby preserving internal validity.
Methods
The paper focuses on propensity score (PS) weighting methods. The core idea is to use the estimated probability of receiving treatment (the propensity score) to re-weight individuals in the dataset, creating a pseudo-population where covariate distributions are more balanced between treatment groups.
Key methods discussed include:
- Inverse Probability Weighting (IPW): The standard approach, which can be unstable with positivity violations.
- Trimming and Truncation: Methods that adjust weights by excluding or capping extreme PS values to mitigate instability.
- Newer Estimands (WATE, WATT, WATC): These generalize standard estimands (ATE, ATT, ATC) by applying "tilting functions" to the weights. These functions are designed to focus on specific subpopulations, such as those with "clinical equipoise" (where treatment assignment is uncertain) or those with better overlap in PS distributions.
- Overlap Weights (OW), Matching Weights (MW), Entropy Weights (EW), Beta Family Weights (BW): These are specific types of tilting functions that aim to improve estimation accuracy and efficiency by down-weighting participants with extreme PS values.
- R Package "ChiPS": The authors developed this package to facilitate the implementation of these methods, offering flexibility in PS model selection (e.g., logistic regression, machine learning).
Datasets
The paper uses two real-world datasets for illustration:
- 2007-2008 National Health and Nutrition Examination Survey (NHANES): Used to study the effect of smoking on blood lead levels. This dataset is a large, nationally representative survey.
- HIV/AIDS prevention program data: Used to study the effect of sex work history on HIV status among transgender women in South Africa. This dataset is from a cross-sectional, community-engaged, non-probability sample.
The paper also uses simulated data generated from various data generating processes (DGPs) to evaluate the performance of different estimators under different scenarios of PS overlap and model misspecification.
Key Insights and Ideas
- Positivity Violations are Common and Problematic: The paper strongly emphasizes that violations of the positivity assumption are not rare and can severely impact the validity of standard causal effect estimates.
- Shifting the Target Population is Key: Instead of trying to estimate effects for the entire population when positivity is violated, the paper advocates for shifting the focus to subpopulations where positivity holds. This is the core idea behind WATE, WATT, and WATC.
- "Weighted" Estimands Offer Robustness: By using methods like overlap weights, researchers can achieve better covariate balance and more stable estimates, especially when positivity is severely violated.
- WATE, WATT, and WATC Provide a Unified Framework: The paper unifies these different classes of estimands, showing how they relate to each other and to standard estimands, providing a structured approach to causal inference.
- The Choice of Estimand Matters: Different weighting methods and estimands can lead to substantially different results, highlighting the importance of careful selection based on the research question and data characteristics.
- R Package Simplifies Complex Methods: The ChiPS package democratizes access to these advanced methods, making them more accessible to researchers.
Key Results
- Simulation Studies: The simulation results demonstrate that methods employing overlap weights (OW, MW, EW) generally exhibit lower relative percent biases (RBias%) and more valid coverage probabilities (CPs) compared to standard IPW, especially under poor PS overlap and model misspecification.
- Real-World Data Application 1 (Smoking & Blood Lead): The analysis using NHANES data showed that the overlap-weighted estimand (ATO) provided excellent covariate balance and stable estimates, suggesting a statistically significant increase in blood lead levels associated with smoking in the overlap population.
- Real-World Data Application 2 (Sex Work & HIV): For the transgender women's study, ATC trimming (a=0.05) was chosen as the primary estimand due to potential positivity violations. This analysis indicated an increased probability of being HIV-positive for those engaging in sex work within the targeted subpopulation. Supplemental analyses showed that while some methods yielded similar results, others (like BW with v=10) showed instability.
Limitations and Future Work
- No Formal Test for Positivity Violations: The authors acknowledge that there are no formal statistical tools to definitively test or identify the type of positivity violation (structural vs. random). Researchers must rely on domain knowledge and graphical diagnostics. Future work could develop such testing procedures.
- PS Model Misspecification: While the paper discusses the impact of PS model misspecification, it notes that even with ensemble learning (like SuperLearner), estimates can still be biased. However, OW, EW, and MW methods are shown to be more robust to misspecification.
- Unconfoundedness and Positivity Connection: The paper assumes unconfoundedness and focuses on positivity violations. Future work could explore the interplay between these two assumptions and develop methods to handle both simultaneously, potentially incorporating sensitivity analyses.
- Augmented Estimators: The tutorial does not include augmented estimators (e.g., doubly robust estimators) which could offer improved efficiency. This was a deliberate choice to focus on PS weights, but future work could explore their integration.
- Covariate Balancing Propensity Score (CBPS): The paper did not discuss CBPS, which directly enforces covariate balance. Future research could investigate how CBPS estimates affect WATE, WATT, and WATC estimands.
- Complex Survey Designs: While survey weights were used in one study, other complex survey design features (strata, PSUs) were not integrated. Methodological developments in this area are needed.
- Data-Driven Selection Caution: The authors strongly caution against data-driven selection of estimands after exploring multiple options, recommending pre-specification in collaboration with experts to ensure principled inference and avoid p-hacking.
Summary updated