Detecting Confounding in Multivariate Linear Models via Spectral Analysis

Dominik Janzing, Bernhard SchölkopfJournal of Causal Inference 2017 · DOI

TL;DR

This paper introduces a novel spectral analysis method to detect and quantify hidden common causes (confounders) in multivariate linear models. By analyzing the "orientation" of regression coefficients relative to the covariance structure of predictors, the method can distinguish between genuine causal influences and spurious correlations, a crucial step for reliable causal inference.

What Problem Does This Paper Address?

Drawing causal conclusions from observational data is notoriously difficult. A key challenge is confounding, where a hidden common cause influences both the observed predictors (X) and the outcome variable (Y), leading to spurious correlations that can be mistaken for genuine causal effects. Traditional methods often struggle to disentangle these effects, especially in high-dimensional settings where direct intervention is impossible and standard conditional independence tests may fail. This paper tackles the problem of quantifying the extent to which observed relationships between a multivariate predictor vector X and a target variable Y are due to direct causation versus a hidden common cause.

Methods

The core of the paper's approach lies in a "principle of generic orientation." The authors propose that in the absence of confounding, the vector of regression coefficients (â) relating Y to X should have a "generic" orientation with respect to the eigenspaces of the covariance matrix of X (Σxx). Confounding, introduced by a hidden common cause Z, is hypothesized to disrupt this generic orientation in a characteristic way.

The method leverages spectral analysis, specifically focusing on the distribution of eigenvalues and eigenvectors of covariance matrices. They define "tracial spectral measures" (related to the distribution of eigenvalues of a matrix) and "vector-induced spectral measures" (related to how a vector aligns with the eigenspaces of a matrix). The key idea is to compare the spectral measure of the observed regression vector â with the expected spectral measure under the assumption of no confounding.

The authors develop a two-parameter model (β, η) to characterize confounding strength and construct a family of spectral measures (νβ,η) that represent different levels of confounding. An algorithm is then proposed to find the parameters (β, η) that best match the observed spectral measure of â, thereby estimating the confounding strength. This involves smoothing spectral measures with a Gaussian kernel and comparing them using an L1 distance.

Datasets

The paper uses simulated data generated according to their model assumptions to evaluate the method's performance across various dimensions (d) and sample sizes (n).

Additionally, they conduct experiments on real-world datasets:

  • Taste of wine dataset: This dataset includes 11 ingredients of red wine and human-rated taste scores.
  • Chicago crime data: This dataset contains crime statistics and socio-economic factors for Chicago community areas.
  • Concrete compressive strength data: This dataset relates the compressive strength of concrete to its constituent materials and age.

Key Insights and Ideas

  • "Generic Orientation" as a Confounder Detector: The central idea is that confounding subtly alters the spectral properties of regression coefficients in a predictable way. This provides a novel angle for detecting hidden causes.
  • Spectral Analysis for Causality: The paper demonstrates the power of spectral analysis (eigenvalue/eigenvector distributions) as a tool for inferring causal structure, going beyond traditional correlation or conditional independence measures.
  • Quantifying Confounding: The introduction of the structural strength of confounding parameter β provides a quantitative measure of how much a hidden common cause distorts the observed relationships.
  • High-Dimensional Advantage: The method's theoretical justification relies on concentration of measure results for high dimensions, suggesting it may be particularly effective in complex, high-dimensional scenarios.

Key Results

  • Simulation Studies: Simulations show that the estimated confounding strength (β) correlates well with the true value, with performance improving with higher dimensions and larger sample sizes. The sample size appears to be more critical than dimension.
  • Real-World Data:
    • On the wine dataset, the method estimated a low confounding strength (β=0) when all ingredients were considered, suggesting they primarily influence taste causally. However, when "alcohol" (the ingredient with the strongest influence) was removed, the estimated confounding strength increased significantly (β=0.55), indicating that its exclusion introduced confounding.
  • For the Chicago crime data, a low confounding strength (β=0.07) was estimated, with unemployment being the strongest predictor of assaults.
  • For the concrete compressive strength data, a high confounding strength (β=0.83) was estimated, suggesting significant influence from hidden common causes.
  • Experimental Setup: Experiments with controlled setups (purely causal and purely confounded scenarios) showed that the algorithm could qualitatively capture the presence of confounding, though it sometimes underestimated its strength in certain configurations.

Limitations and Future Work

  • Idealized Model: The theoretical justification relies on a highly idealized model where parameters are randomly generated from a rotation-invariant prior. While empirical data might exhibit similar concentration phenomena, this model is likely violated in practice.
  • High-Dimensional Assumption: The method's theoretical underpinnings are strongest for high dimensions. Its performance in low-dimensional settings might be less robust.
  • Underestimation in Real Experiments: In controlled experiments, the algorithm tended to underestimate confounding, possibly because the causal vector 'a' was not in a "generic" orientation relative to Σxx. This non-genericness of 'a' could mask the effects of confounding.
  • Normalization Sensitivity: The authors note that normalization of variables can improve robustness in simulations but may conflict with the method's theoretical justification. They recommend avoiding normalization if possible.
  • Future Work: The authors suggest exploring the application of the method with regularization techniques and investigating its behavior with data exhibiting multivariate total positivity (MTP2). They also acknowledge the difficulty of precisely quantifying causal influence and the need for further research in this area.

Summary updated

Abstract

Abstract We study a model where one target variable <m:math xmlns:m="http://www.w3.org/1998/Math/MathML"> <m:mi>Y</m:mi> </m:math> $Y$ is correlated with a vector <m:math xmlns:m="http://www.w3.org/1998/Math/MathML"> <m:mrow> <m:mrow> <m:mtext mathvariant="bold">X</m:mtext> </m:mrow> </m:mrow> <m:mo>:=</m:mo> <m:mo stretchy="false">(</m:mo> <m:msub> <m:mi>X</m:mi> <m:mn>1</m:mn> </m:msub> <m:mo>,</m:mo> <m:mo>…</m:mo> <m:mo>,</m:mo> <m:msub> <m:mi>X</m:mi> <m:mi>d</m:mi> </m:msub> <m:mo stretchy="false">)</m:mo> </m:math> $\textbf{X}:=(X_1,\dots,X_d)$ of predictor variables being potential causes of <m:math xmlns:m="http://www.w3.org/1998/Math/MathML"> <m:mi>Y</m:mi> </m:math> $Y$ . We describe a method that infers to what extent the statistical dependences between <m:math xmlns:m="http://www.w3.org/1998/Math/MathML"> <m:mrow> <m:mrow> <m:mtext mathvariant="bold">X</m:mtext> </m:mrow> </m:mrow> </m:math> $\textbf{X}$ and <m:math xmlns:m="http://www.w3.org/1998/Math/MathML"> <m:mi>Y</m:mi> </m:math> $Y$ are due to the influence of <m:math xmlns:m="http://www.w3.org/1998/Math/MathML"> <m:mrow> <m:mrow> <m:mtext mathvariant="bold">X</m:mtext> </m:mrow> </m:mrow> </m:math> $\textbf{X}$ on <m:math xmlns:m="http://www.w3.org/1998/Math/MathML"> <m:mi>Y</m:mi> </m:math> $Y$ and to what extent due to a hidden common cause (confounder) of <m:math xmlns:m="http://www.w3.org/1998/Math/MathML"> <m:mrow> <m:mrow> <m:mtext mathvariant="bold">X</m:mtext> </m:mrow> </m:mrow> </m:math> $\textbf{X}$ and <m:math xmlns:m="http://www.w3.org/1998/Math/MathML"> <m:mi>Y</m:mi> </m:math> $Y$ . The method relies on concentration of measure results for large dimensions <m:math xmlns:m="http://www.w3.org/1998/Math/MathML"> <m:mi>d</m:mi> </m:math> $d$ and an independence assumption stating that, in the absence of confounding, the vector of regression coefficients describing the influence of each <m:math xmlns:m="http://www.w3.org/1998/Math/MathML"> <m:mrow> <m:mrow> <m:mtext mathvariant="bold">X</m:mtext> </m:mrow> </m:mrow> </m:math> $\textbf{X}$ on <m:math xmlns:m="http://www.w3.org/1998/Math/MathML"> <m:mi>Y</m:mi> </m:m
BibTeX
@article{janzing2017detecting,
  author = {Dominik Janzing and Bernhard Schölkopf},
  title = {{Detecting Confounding in Multivariate Linear Models via Spectral Analysis}},
  journal = {Journal of Causal Inference},
  year = {2017},
  doi = {10.1515/jci-2017-0013},
}
Figure