TL;DR
This paper tackles a common problem in causal inference: when the "positivity assumption" is violated, standard methods for estimating treatment effects can produce unreliable results. The authors propose a new method, "Overlap Weighted Average Treatment Effect on the Treated" (OWATT), which offers a more robust and data-driven way to estimate treatment effects, especially for the "treated" group, without requiring arbitrary user-defined thresholds.
What Problem Does This Paper Address?
In causal inference, we often want to understand the effect of a treatment (or intervention) on an outcome. Propensity scores (PS) are a popular tool to adjust for confounding factors in observational studies. A key assumption for PS methods is "positivity," which essentially means that every individual has a non-zero probability of receiving any treatment. When this assumption is violated (i.e., some individuals have a very high or very low probability of treatment), it leads to extreme weights in the PS analysis, which can make the results unstable and unreliable.
While methods like trimming or truncating extreme PS weights exist to address this, they require researchers to arbitrarily choose a threshold beforehand. This paper focuses on estimating the "Average Treatment Effect on the Treated" (ATT), which is particularly relevant for understanding the impact of a treatment on those who actually received it. The authors highlight that while there are solutions for estimating the Average Treatment Effect (ATE) under positivity violations, there's a lack of robust methods specifically for ATT. This gap means researchers might be forced to use ad-hoc methods that can lead to subjective results.
Methods
The paper introduces a novel approach called Overlap Weighted Average Treatment Effect on the Treated (OWATT). This method builds upon the concept of "overlap weights" (OW) previously proposed for ATE estimation. The core idea is to use a "tilting function" that smoothly down-weights control participants whose PSs are at the extremes, rather than abruptly removing them (trimming) or capping their weights (truncation).
Specifically, OWATT uses a tilting function $h(x) = e(x)\{1 - e(x)\}$, where $e(x)$ is the propensity score. This function is designed to target a population of control participants for whom there is "clinical equipoise" – meaning they have a high probability of receiving either treatment. This approach avoids the need to pre-specify a threshold, making it more data-driven and less subjective. The authors also review and discuss existing trimming and truncation methods for ATT, highlighting their limitations.
Datasets
The authors use two types of data for their evaluation:
- Simulated Data: They generate data based on two distinct data-generating processes (DGPs) that mimic scenarios with varying degrees of "overlap" (i.e., how similar the PS distributions are between treatment and control groups) and different model specifications (correctly specified vs. misspecified PS models). This allows for a controlled comparison of their proposed method against existing ones.
- Real-World Data: They analyze data from the Medical Expenditure Panel Survey (MEPS) to investigate racial disparities in health care expenditure between White (treated group) and Asian (control group) participants. This application demonstrates the practical utility of their method.
Key Insights and Ideas
- The "Positivity Problem" is Real and Affects ATT: Violations of the positivity assumption are not just theoretical; they can lead to unstable and biased estimates of the ATT, a crucial estimand for policy and research.
- Trimming/Truncation is Ad-Hoc: Existing methods like trimming and truncation require researchers to make subjective choices about thresholds, which can significantly impact results and potentially lead to "cherry-picking" findings.
- OWATT Offers a Smoother, Data-Driven Solution: The proposed OWATT method uses a continuous weighting function that naturally down-weights extreme PSs, avoiding the abruptness of trimming and the subjectivity of threshold selection.
- Clinical Equipoise as a Target: The tilting function used in OWATT aims to focus on participants with a high probability of receiving either treatment, mimicking the conditions of a randomized trial and providing a clinically meaningful target population.
- OWATT is Robust: Simulation results suggest OWATT performs well across various scenarios, including when the PS model is misspecified, demonstrating its robustness.
Key Results
- Simulation Studies:
- In simulations, OWATT consistently showed the smallest Root Mean Square Error (RMSE) and highest coverage probability (CP) close to the nominal 95% level when the PS model was correctly specified, indicating high internal validity and efficiency.
- Even when the PS model was misspecified, OWATT generally outperformed other methods in terms of RMSE, demonstrating its robustness.
- ATT trimming and truncation methods showed less consistent patterns and were more sensitive to the choice of thresholds.
- MEPS Data Analysis:
- The OWATT estimator provided a point estimate of $2511.91 for the average health care expenditure difference between White and Asian participants.
- OWATT achieved better covariate balance compared to standard ATT and some other ad-hoc methods, as visualized in the love plot. This suggests that OWATT effectively balances the distributions of covariates between the groups.
- The standard error for OWATT was notably smaller than for ATT, indicating greater precision.
Limitations and Future Work
- PS Model Misspecification: The authors acknowledge that all methods, including OWATT, are biased when the PS model is misspecified. They recommend using saturated PS models or machine learning techniques to improve PS estimation.
- Structural Positivity Violations: OWATT helps with "random" positivity violations (due to estimation) but doesn't solve "structural" violations where the true PS distribution inherently lacks positivity.
- Future Work: The authors plan to investigate:
- The semiparametric efficiency of WATT estimators.
- The use of other tilting functions (e.g., Shannon's entropy, matching functions).
- Extending the framework to estimate the Average Treatment Effect on the Control (ATC).
- The theoretical similarity between OWATT and ATT when overlap is good or moderate.
Summary updated