Proportion-based Sensitivity Analysis of Uncontrolled Confounding Bias in Causal Inference

Haruka Yoshida, Manabu KurokiIJCAI 2024

Abstract

In observational studies, uncontrolled confounding bias remains a primary threat to the validity of causal conclusions. This paper introduces Proportion-based Sensitivity Analysis (PSA), a novel framework that quantifies the robustness of causal claims by framing the sensitivity parameter as a proportion of the total mean squared error (MSE). Unlike traditional methods that require specific assumptions about the nature of hidden variables, PSA provides intuitive bounds and a calculable "cutoff value" to determine if the sign of an observed effect is resilient to hidden biases.

Introduction: The Challenge of Unmeasured Confounders

While randomized controlled trials are the gold standard for establishing causality, researchers often rely on observational data where uncontrolled confounding is a persistent risk. This occurs when unmeasured variables influence both the treatment and the outcome, potentially creating spurious correlations. Current sensitivity analysis tools often demand restrictive assumptions—such as the hidden variable being binary or having a specific correlation strength—which limits their generalizability. PSA addresses this by treating total bias as a single, interpretable parameter.

Methodology

The core of the PSA method is the Proportion-based Sensitivity Parameter (PSP). The authors demonstrate that the Mean Squared Error (MSE) of an outcome variable can be decomposed into conditional variance (inherent noise) and the square of the uncontrolled confounding bias. The PSP is defined as the ratio of this squared bias to the total MSE, resulting in a value between 0 and 1 that represents the "percentage of error due to confounding."

The framework provides two primary metrics:

  • w-PSP: Applied to the whole population.
  • s-PSP: Applied to specific subpopulations, such as those receiving a specific treatment.

A critical feature of this method is the cutoff value. This estimable point, derived from observed data, indicates the maximum proportion of confounding bias that can exist before the sign of the causal effect (e.g., whether a drug is beneficial or harmful) would flip. If the estimated PSP is below this cutoff, the researcher can maintain confidence in the direction of their causal claim.

Empirical Evaluation

The authors validated PSA using two distinct real-world datasets: an industrial study on car body coating efficiency and a medical study from the Cooperative Cardiovascular Project.

Medical Case Study: Beta-Blockers and Mortality

The medical dataset examined the impact of beta-blockers on 30-day mortality in heart attack patients, stratified by ethnicity. The raw data for this study is summarized in the table below:

Table 1: Data on 30-day mortality for heart attack patients treated with beta-blockers, stratified by race.

In the analysis of black patients, PSA identified a cutoff value of 10.7%. This implies that unless unmeasured confounding accounts for more than 10.7% of the total MSE, the preventive effect of beta-blockers remains statistically valid.

Key Results and Comparison

A significant advantage of PSA is its ability to provide much tighter bounds than existing methodologies. When compared to the "Tian-Pearl bounds," which are often too wide to offer actionable insights in practical scenarios, PSA provides a significantly more constrained and useful range for researchers.

Figure 1: Comparison between PSA Bounds and Tian-Pearl Bounds relative to the s-psp parameter.

As shown in Figure 1, the PSA bounds (the shaded region) are considerably narrower than the traditional Tian-Pearl bounds, especially as the proportion of confounding (s-psp) increases. This precision allows practitioners to make more informed decisions about the robustness of their findings.

Discussion and Limitations

While PSA offers a minimalist and intuitive approach to sensitivity analysis, it does have limitations. The linear Structural Equation Model (SEM) components currently assume Gaussian distributions and linear relationships, which may not capture the complexity of all systems. Additionally, in some instances, researchers must still assume the direction of the bias to pinpoint the exact causal effect. Future work aims to extend these 2D visualizations and formulas to high-dimensional models involving multiple interacting treatments.

Summary updated

BibTeX
@inproceedings{yoshida2024proportion,
  author = {Haruka Yoshida and Manabu Kuroki},
  title = {{Proportion-based Sensitivity Analysis of Uncontrolled Confounding Bias in Causal Inference}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2024},
}
Figure