Abstract
Estimating causal effects in high-dimensional datasets remains a significant challenge, particularly when traditional confounders are unobserved or when severe multicollinearity exists. This paper introduces the Penalized Covariate-Mediator Selection Operator (PCM Selector), a two-stage penalized regression method that integrates causal graph theory with regularization techniques. By utilizing a front-door-like criterion and specialized $L_p$-penalized loss functions, the PCM Selector provides consistent and accurate causal estimates where standard methods like LASSO often fail.
Introduction
In causal inference, the "back-door criterion" is the standard approach for identifying causal relationships: if all variables influencing both the treatment ($X$) and the outcome ($Y$) are controlled, the true effect can be isolated. However, real-world data often presents two major hurdles: unobserved confounders and the curse of dimensionality. High-dimensional data frequently suffers from multicollinearity, causing standard tools like Ordinary Least Squares (OLS) to produce unstable estimates. While machine learning tools like LASSO address dimensionality, they are often "causally blind," prioritizing predictive power over causal accuracy and potentially dropping essential variables.
Methodology: The PCM Selector
The PCM Selector shifts the focus from back-door covariates to intermediate variables, or mediators, that lie on the causal path between the treatment and the outcome. This approach leverages "front-door-like" logic to identify causal effects even when confounders are unobserved.
Key technical components include:
- Two-Stage Penalized Regression: Unlike standard single-penalty methods, the PCM Selector uses a specialized $L_p$-penalized loss function. It incorporates prior causal knowledge to ensure that known causal path variables are not erroneously shrunk to zero.
- Adaptive Weighting: Building on insights from D.R. Cox (1960), the method employs standardized weight vectors to prioritize covariates and mediators that maximize estimation accuracy.
- Bias Reduction: A modified version of the estimator is designed specifically to mitigate the bias typically introduced by $L_1$ (LASSO-style) penalties.
Experimental Evaluation
The authors evaluated the PCM Selector using synthetic numerical experiments based on Linear Structural Causal Models (SCMs). The study involved 15 random samples of 18 variables (10 covariates, 5 mediators) across 5,000 replications. The PCM Selector was benchmarked against industry standards, including LASSO, adaptive LASSO, Elastic Net, OLS, and $PAL_pMA$.
Results and Discussion
The simulations demonstrated that the PCM Selector significantly outperforms traditional methods in both stability and accuracy. One key insight is that including mediators can actually improve the precision of the total effect estimate compared to using back-door covariates alone.
Major findings include:
- Lower Bias: In "Setting (b)" (unobserved covariates), the PCM Selector achieved a bias of 0.046, notably lower than the 0.066 bias of the minimal Front-door method.
- Directional Accuracy: The PCM Selector maintained a high "sign coincidence rate" (between 0.526 and 0.808). In contrast, LASSO and Elastic Net often failed to identify the correct direction of the causal effect, with sign coincidence rates dropping as low as 0.117.
- Statistical Efficiency: Theorems presented in the paper prove that the PCM Selector's variance is approximately lower than or equal to OLS-based estimates, ensuring a more stable result in high-dimensional settings.
Conclusion and Future Work
The PCM Selector successfully bridges the gap between graph-based causal identification and high-dimensional regularization. While currently limited to linear structural equation models and requiring some prior causal knowledge of the graph structure, it represents a significant step forward for causal discovery. Future research aims to extend this framework to Generalized Linear Models (GLMs) to accommodate binary and count outcomes.
Summary updated