MERLiN: Mixture Effect Recovery in Linear Networks

Sebastian Weichwald, Moritz Grosse-Wentrup, Arthur GrettonOpenReview 2022 · OpenReview

TL;DR

This paper introduces MERLIN, a novel family of algorithms for causal inference that constructs meaningful "causal variables" from observed, mixed signals. MERLIN tackles the challenge of identifying cause-effect relationships when the true underlying variables are not directly observable, but rather appear as linear combinations of measured data, particularly in neuroimaging.

What Problem Does This Paper Address?

In many scientific domains, especially neuroimaging like EEG, we observe signals that are mixtures of underlying, more meaningful "causal" variables. For instance, EEG electrodes pick up a mix of electrical activity from different brain regions. To understand brain function, we're interested in the causal relationships between these underlying brain regions, not the raw electrode signals. Traditional causal inference methods often assume these causal variables are directly observable or require complex interventions. This paper addresses the problem of inferring causal relationships when the observed data is a linear mixture of these unobserved causal variables, and we need to first construct these causal variables before we can infer relationships between them.

Methods

The core innovation is MERLIN (Mixture Effect Recovery in Linear Networks), a family of algorithms designed to recover a causal variable that is a linear effect of another known causal variable, even when observed through a linear mixture. MERLIN works by optimizing a specific objective function that captures statistical properties implying a cause-effect relationship.

Key aspects of the method include:

  • Causal Variable Construction: Instead of assuming pre-defined causal variables, MERLIN actively constructs them.
  • Optimization-Based Approach: The algorithms optimize a function that encourages statistical dependencies indicative of a causal link, specifically aiming to find a linear combination of observed signals that acts as a causal effect.
  • Precision Matrix Magic (MERLIN_Σ-1): A core algorithm leverages the relationship between conditional independences in Gaussian distributions and zero entries in the precision matrix (inverse covariance matrix). By optimizing the precision matrix entries, MERLIN can identify causal effects.
  • Adaptation for Time-Series Data (MERLIN_bp): For data like EEG, MERLIN is extended to handle time-series by incorporating log-bandpower computation, allowing it to identify causal relationships within specific frequency bands.
  • Incorporating Prior Knowledge (MERLIN_bp+): The framework can be further refined by incorporating domain knowledge, such as the properties of volume conduction in EEG, to improve the accuracy and interpretability of the recovered causal variables.
  • Orthogonal Mixing Assumption: To mitigate issues where the target causal variable might be indistinguishable from other mixed signals, the authors assume an orthogonal mixing matrix, which effectively "removes" the influence of the known cause from the observed mixture.

Datasets

The paper uses two types of datasets:

  • Synthetic Datasets: The authors generate synthetic data to rigorously test their algorithms under controlled conditions. These datasets are based on structural equation models (SEMs) with varying parameters for noise levels, dimensionality, and confounding.
  • Empirical EEG Data: The algorithms are evaluated on real EEG data recorded during a neurofeedback experiment. This data involves subjects attempting to up- or down-regulate gamma-oscillations in a specific brain region (superior parietal cortex, SPC). The data includes stimulus information (instruction to up/down-regulate), electrode signals, and pre-computed spatial filters.

Key Insights and Ideas

  • Causal Inference Requires Causal Variable Construction: The paper strongly argues that standard causal inference methods are insufficient when the observed variables are mere mixtures. The ability to construct meaningful causal variables is a prerequisite.
  • Statistical Properties as Causal Indicators: MERLIN's core idea is that specific statistical dependencies (or independencies) between variables, when optimized for, can directly imply a causal structure, even without explicit interventions.
  • Beyond Source Separation: Unlike methods like Independent Component Analysis (ICA), which aim to separate mixed signals without necessarily considering causal structure, MERLIN explicitly targets causal relationships.
  • EEG as a Prime Application: The paper highlights EEG as a domain where MERLIN is particularly relevant due to the inherent linear mixing of signals and the interest in understanding functional brain networks.
  • The Power of Imaginary Coherency: For EEG, incorporating imaginary coherency as a prior knowledge constraint helps to distinguish true causal interactions from volume conduction artifacts.

Key Results

  • Successful Causal Variable Recovery: MERLIN algorithms demonstrated good performance on both synthetic and empirical EEG data, successfully recovering causal variables that were effects of a known causal variable.
  • Robustness to Confounding: The MERLIN_Σ-1 algorithm showed robustness to hidden confounding in synthetic data.
  • Performance Degradation with Complexity: Performance generally decreased with higher dimensionality and increased noise levels in synthetic experiments, but remained probabilistically sound.
  • EEG Application Success: In the EEG neurofeedback experiment, MERLIN_bp+ successfully identified a brain region (medial prefrontal cortex, MPFC) as a causal effect of the targeted brain region (SPC), aligning with previous hypotheses and findings.
  • Efficiency: MERLIN is computationally efficient, bypassing the need for extensive source localization or exhaustive searches over many potential brain regions.

Limitations and Future Work

  • Linearity Assumption: The current MERLIN algorithms are designed for linear networks and cause-effect relationships. The authors acknowledge this limitation and suggest future work on extending MERLIN to non-linear scenarios.
  • Faithfulness Assumption: Like many causal inference methods, MERLIN relies on the faithfulness assumption (that all independencies are implied by the causal graph). While untestable, the authors note that MERLIN can handle some unfaithful scenarios and that future research could explore robustness to violations.
  • Recovering Multiple Effects: MERLIN, in its basic form, recovers a single causal variable. Disentangling multiple distinct effects from a mixture requires incorporating prior knowledge or further algorithmic extensions.
  • High-Dimensional Data: For very high-dimensional data (d > m), additional regularization is needed to prevent overfitting and ensure stable recovery.
  • Future Directions: The authors suggest extending MERLIN to other neuroimaging modalities like fMRI, developing methods for assessing statistical significance and uncertainty of recovered variables, and exploring non-linear extensions.

Summary updated

Abstract

Causal inference concerns the identification of cause-effect relationships between variables, e. g. establishing whether a stimulus affects activity in a certain brain region. The observed variables themselves often do not constitute meaningful causal variables, however, and linear combinations need to be considered. In electroencephalographic studies, for example, one is not interested in establishing cause-effect relationships between electrode signals (the observed variables), but rather between cortical signals (the causal variables) which can be recovered as linear combinations of electrode signals. We introduce MERLiN (Mixture Effect Recovery in Linear Networks), a family of causal inference algorithms that im- plement a novel means of constructing causal variables from non-causal variables. We demonstrate through application to EEG data how the basic MERLiN algorithm can be extended for application to different (neuroimaging) data modalities. Given an observed linear mixture, the algorithms can recover a causal variable that is a linear effect of another given vari- able. That is, MERLiN allows us to recover a cortical signal that is affected by activity in a certain brain region, while not being a direct effect of the stimulus. The Python/Matlab implementation for all presented algorithms is available on https://github.com/sweichwald/MERLiN.
BibTeX
@misc{weichwald2022merlin,
  author = {Sebastian Weichwald and Moritz Grosse-Wentrup and Arthur Gretton},
  title = {{MERLiN: Mixture Effect Recovery in Linear Networks}},
  year = {2022},
  url = {https://openreview.net/forum?id=rjbcWJQ64dJ},
  howpublished = {OpenReview},
}
Figure