Dependency in DAG models with hidden variables

Robin J. Evans

Introduction

In causal inference, Directed Acyclic Graphs (DAGs) are the standard tool for representing how variables influence one another. However, real-world systems often contain hidden or latent variables that complicate these relationships. When such variables are present, we typically transition to Acyclic Directed Mixed Graphs (ADMGs). A fundamental assumption in these models is that if two variables do not share a direct edge or a common hidden parent, they should not be able to "mimic" each other perfectly without the involvement of the rest of the network.

This paper explores a counterintuitive phenomenon: can two observed variables, $X_a$ and $X_c$, be identically equal ($X_a = X_c$) while remaining jointly independent of all other observed variables? The author demonstrates that this is indeed possible, suggesting that standard interpretations of independence in graphs may overlook a layer of complexity known as nested constraints.

The Instrumental Variables Surprise

A primary example of this phenomenon occurs in the standard Instrumental Variables (IV) model. In this setup, a variable $a$ influences $b$, which in turn influences $c$, while a hidden variable $h$ influences both $b$ and $c$.

The Instrumental Variables (IV) model represented as (a) a DAG with a latent variable $h$ and (b) an ADMG with a bidirected edge.

Despite $X_b$ sitting between $X_a$ and $X_c$, and despite $X_a$ and $X_c$ sharing no direct edge or latent parent, the author proves it is possible to construct functions such that $X_a = X_c$ exactly. This "perfect correlation" occurs while both variables remain independent of $X_b$, a result that challenges traditional d-separation logic.

Theoretical Foundations and "Densely Connected" Vertices

The paper identifies the precise graphical condition required for this perfect correlation: densely connected vertices. Two vertices are considered densely connected if they share a specific ancestral relationship or if their "intrinsic closures" are bidirected-connected.

Using a constructive mathematical approach involving modular arithmetic (XOR gates), the author establishes Theorem 6.4. This core result proves that any joint distribution on $(X_v, X_w)$ can be obtained independently of all other variables if and only if $v$ and $w$ are adjacent in the maximal arid projection of the graph. This finding reinforces the nested Markov property as the superior framework for understanding constraints in latent variable models.

Empirical Evidence and Simulations

To validate these theoretical results, the author performed synthetic simulations. While the proofs often rely on binary variables and XOR logic, the simulations demonstrate that these dependencies extend to continuous normal distributions using bitwise operations and structural equations.

Scatter plot matrix demonstrating the theoretical "perfect correlation" between $X_a$ and $X_c$ (diagonal line) while both remain independent of $X_b$ (circular blobs).

As shown in the scatter plots, $X_a$ and $X_c$ exhibit a perfect linear relationship, yet their joint distribution with $X_b$ shows no correlation, confirming the independence predicted by the theory.

Implications for Causal Discovery

The paper provides a linear-time $O(|V|)$ algorithm to identify the minimal subgraph needed to create these dependencies. This has significant implications for causal discovery: researchers can simplify their search space by focusing on "maximal arid graphs" (MArGs) without losing the ability to represent complex distributions.

While the current work focuses on pairs of variables, future research is needed to scale these findings to larger groups of variables, as pairwise equality does not always imply simultaneous equality across a set while maintaining independence from the rest of the system.

Summary updated

Abstract

Directed acyclic graph models with hidden variables have been much studied, particularly in view of their computational efficiency and connection with causal methods. In this paper we provide the circumstances under which it is possible for two variables to be identically equal, while all other observed variables stay jointly independent of them and mutually of each other. We find that this is possible if and only if the two variables are `densely connected'; in other words, if applications of identifiable causal interventions on the graph cannot (non-trivially) separate them. As a consequence of this, we can also allow such pairs of random variables have any bivariate joint distribution that we choose. This has implications for model search, since it suggests that we can reduce to only consider graphs in which densely connected vertices are always joined by an edge.
BibTeX
@inproceedings{evans2021dependency,
  author = {Robin J. Evans},
  title = {{Dependency in DAG models with hidden variables}},
  booktitle = {Conference on Uncertainty in Artificial Intelligence},
  year = {2021},
}
Figure