Noisy-OR Models with Latent Confounding

Antti Hyttinen, Frederick Eberhardt, Patrik O. Hoyer

TL;DR

This paper introduces a new class of causal graphical models called "noisy-OR models with latent confounding" for binary variables. They provide a theoretical framework and learning algorithms that can identify the full causal structure and parameters from a combination of observational data and experiments, even when hidden common causes (confounders) are present. This is a significant step towards robust causal discovery in complex systems where full experimental control is often infeasible.

What Problem Does This Paper Address?

Understanding causal relationships is crucial for controlling and predicting the behavior of complex systems. While observational data can reveal statistical dependencies, it's often insufficient to disentangle true causal effects from spurious correlations induced by unobserved common causes (latent confounding). Randomized experiments are powerful for causal inference, but intervening on many variables simultaneously can be prohibitively expensive. This paper tackles the challenge of identifying causal models, including the presence of latent confounders, using a combination of passive observational data and experiments that intervene on only one variable at a time. They specifically address the limitations of existing methods when dealing with discrete variables and arbitrary causal relationships, proposing a solution within the constrained framework of noisy-OR models.

Methods

The core of the paper lies in defining and identifying a "noisy-OR model with latent confounding." This model uses a specific parametric form for the conditional probability distributions of binary variables, where a variable's state is determined by a logical OR of its parents' influences, plus a disturbance term. Crucially, latent confounding is modeled by allowing these disturbance terms to be dependent.

The paper then develops a four-step identification procedure:

  1. Causal Ordering: Establishing a partial causal order among variables by comparing intervention outcomes.
  2. Adjacent Edge Coefficients: Identifying direct causal influences between adjacent variables in the causal order using a "causal power" statistic.
  3. Non-Adjacent Edge Coefficients: Generalizing the causal power statistic to infer influences between non-adjacent variables by conditioning on intermediate variables. This step leverages a key property called "context-specific independence" inherent in the noisy-OR structure.
  4. Disturbance Distribution: Inferring the distribution of the latent confounders from the identified model and observational data.

Two learning algorithms are proposed:

  • EC-algorithm: An efficient algorithm that selects minimal conditioning sets to reduce the burden of excessive conditioning, aiming for sparsity.
  • EM-algorithm: A maximum likelihood estimation approach that utilizes all available data, particularly effective for smaller models but computationally intensive for larger ones.

The model is also extended to handle "negative causes," where a parent can decrease the probability of its child's activation, making the model more flexible.

Datasets

The paper does not introduce new datasets. Instead, it relies on simulated data generated from synthetic noisy-OR models (both with and without confounding, and with and without negative causes) to evaluate the performance of their proposed algorithms. These simulations are designed to test accuracy, scalability, and robustness under various conditions.

Key Insights and Ideas

  • Identifiability under Weak Assumptions: The paper demonstrates that full causal model identification (structure and parameters) is possible even with latent confounding, using only passive observational data and single-variable interventions, provided the noisy-OR parametric assumption holds. This is a significant relaxation of assumptions compared to many existing methods.
  • Context-Specific Independence is Key: The noisy-OR structure's property of context-specific independence (specifically, conditioning on a child being OFF blocks indirect paths) is crucial for inferring causal links when latent confounders are present. This property is not shared by all causal models.
  • Noisy-OR as a Bridge: The noisy-OR model serves as a bridge between the simplicity of linear models (where identifiability results are well-established) and the complexity of discrete models with arbitrary dependencies, offering a tractable yet powerful framework.
  • Negative Causes Extend Flexibility: The generalization to negative causes significantly broadens the applicability of the noisy-OR model, allowing it to represent a wider range of real-world causal interactions.

Key Results

  • Theoretical Guarantee: Theorem 1 establishes that the proposed noisy-OR model with latent confounding is identifiable from a specific set of experiments (passive observation plus single-variable interventions for each ordered pair).
  • Algorithm Performance:
    • The EM-algorithm achieves high accuracy (around 0.9) for estimating link probabilities with relatively small sample sizes (1,000 total samples for an 8-node graph). However, it is computationally infeasible for larger structures.
  • The EC-algorithm scales well to larger numbers of variables. For models with up to 20 variables, it achieves reasonable accuracy in terms of edge additions and deletions, with false positive rates around 20% and false negative rates also around 20% under certain conditions.
  • Robustness: The paper shows that the noisy-OR model performs better than Bayesian networks when confounding is present but interactions are absent, while Bayesian networks excel when interactions are present and confounding is absent. Both models struggle when their respective model space assumptions are violated.

Limitations and Future Work

  • Parametric Assumption: The identifiability results rely on the noisy-OR parametric assumption. If the true underlying causal mechanism deviates significantly from this, the methods may not be applicable.
  • Computational Complexity of EM: The EM-algorithm, while accurate, is computationally infeasible for large models due to the exponential number of states for disturbance variables.
  • Future Work: The authors suggest several promising directions:
  • Investigating stronger priors on disturbance distributions to improve estimation.
  • Characterizing what can be learned when identifiability conditions are not met.
  • Extending the approach to cyclic (non-recursive) models.
  • Exploring further extensions to models with interactive causal effects.

Summary updated

Abstract

Given a set of experiments in which varying subsets of observed variables are subject to intervention, we consider the problem of identifiability of causal models exhibiting latent confounding. While identifiability is trivial when each experiment intervenes on a large number of variables, the situation is more complicated when only one or a few variables are subject to intervention per experiment. For linear causal models with latent variables Hyttinen et al. (2010) gave precise conditions for when such data are sufficient to identify the full model. While their result cannot be extended to discrete-valued variables with arbitrary cause-effect relationships, we show that a similar result can be obtained for the class of causal models whose conditional probability distributions are restricted to a `noisy-OR' parameterization. We further show that identification is preserved under an extension of the model that allows for negative influences, and present learning algorithms that we test for accuracy, scalability and robustness.
BibTeX
@inproceedings{hyttinen2011noisy,
  author = {Antti Hyttinen and Frederick Eberhardt and Patrik O. Hoyer},
  title = {{Noisy-OR Models with Latent Confounding}},
  booktitle = {Conference on Uncertainty in Artificial Intelligence},
  year = {2011},
}
Figure