In the world of causal inference, the Common Cause Principle (CCP) is a cornerstone: if two events are correlated but neither causes the other, there must be a shared history—a common cause—that explains their connection. But what happens when that cause is "latent" (hidden)? If we only see the effects, how do we reconstruct the most plausible version of the cause?
In their recent paper, Hovhannisyan and Allahverdyan tackle this "latent confounder" problem using a framework borrowed from statistical mechanics and information theory. They uncover a fascinating phenomenon: the "most likely" cause doesn't just change smoothly as data changes; it can undergo "phase transitions" similar to those found in physical matter.
TL;DR
The authors propose using a generalized maximum likelihood approach to identify the most probable latent common cause between variables. They demonstrate that as the observed correlation shifts from positive to negative, the underlying causal structure undergoes a non-analytic "second-order phase transition," and they show that standard graph-based models (DAGs) may be insufficient to represent these optimal causal structures.
What Problem Does This Paper Address?
Causal analysis often hits a wall called causal insufficiency. This occurs when we know a common cause $C$ exists for variables $A$ and $B$, but we can only observe the joint probability $p(a, b)$. Because $C$ is hidden, the system is "non-identifiable"—meaning there are infinitely many mathematical ways to define $C$ that fit the observed data.
The challenge is to find a principled way to pick the "best" or "most likely" $C$. While previous methods often favored "sparse" or deterministic causes, this paper seeks a method consistent with the broader methodological spirit of the CCP, particularly in how it relates to predictive power and entropy.
Methods
The authors employ the Generalized Maximum Likelihood (GL) method.
- The GL Function: They define a likelihood function $L_\beta(\theta)$ that incorporates a hyperparameter $\beta$.
- Maximum Entropy Connection: When $\beta \lesssim 1$, the maximization of this likelihood becomes equivalent to maximizing the conditional entropy $H(C|AB)$. This aligns the search for a common cause with the Maximum Entropy principle—choosing the cause that assumes the least amount of "extra" information not present in the data.
- Numerical Maximization: For complex cases (like three observed variables), they use numerical optimization to find the parameters of the common cause that maximize this generalized likelihood under the constraints of the observed joint distribution.
- Symmetry Analysis: They specifically analyze symmetric binary variables to see how the "most likely" cause behaves as the correlation between $A$ and $B$ is tuned.
Datasets
The paper is primarily theoretical and methodological. It does not use large-scale empirical datasets. Instead, it relies on:
- Synthetic Binary Models: Mathematical constructions of two and three binary variables.
- Reichenbach’s Geyser Example: A classic causal inference thought experiment involving two geysers that erupt together due to a shared underground plumbing system. The authors use this to provide concrete numerical results for their GL approach.
Key Insights and Ideas
- Causal Phase Transitions: This is the "aha moment" of the paper. In the binary case, as the observed distribution moves from correlation (A and B happen together) to anti-correlation (A happens when B doesn't), the conditional probabilities of the common cause change non-analytically. The first derivatives remain continuous, but the behavior is reminiscent of a second-order phase transition in physics.
- Symmetry Persistence: If the observed effects $A$ and $B$ are symmetric, the most likely common cause $C$ will also act on them symmetrically.
- The "Better Predictor" Property: The authors argue that a true common cause should be a better predictor of the effects than the effects are of each other. Their GL method naturally satisfies this, whereas alternative "minimum entropy" methods often fail this test by producing causes that are too deterministic.
- DAG Defiance: When dealing with three observed variables, the "most likely" common cause can result in causal structures that cannot be represented by standard Directed Acyclic Graphs (DAGs) that satisfy the Markov condition. This suggests that the "most likely" reality might be more complex than our standard graphical tools can capture.
Key Results
- Consistency with Independence: The authors prove that if $A$ and $B$ are independent, the GL method correctly predicts "no cause"—specifically, it yields a uniform distribution for $C$ that provides no predictive power, as expected.
- Geyser Numerical Solution: For a specific correlated "geyser" distribution, the method identifies a rare, quasi-deterministic common cause ($p(c) \approx 0.11$) that almost perfectly triggers both effects ($p(a|c) = p(b|c) \approx 0.94$).
- Critique of Sparsity: They demonstrate that "Predictive Likelihood" and "Minimum Common Entropy" methods tend to force probabilities to zero or one (sparsity). They argue this is often an artifact of the math rather than a reflection of reality, requiring "fine-tuned" or deterministic assumptions that aren't always justified.
Limitations and Future Work
- Dimensionality of C: The authors largely assumed the number of states for the hidden cause $C$ is known a priori. Future work is needed to determine the dimensionality of $C$ automatically.
- Scaling Up: The current analysis focuses on binary or small discrete sets. Extending this to variables with large discrete domains or continuous domains (like Gaussian variables) is a key next step.
- Computational Complexity: While GL is theoretically sound, finding global maxima in non-convex optimization problems (especially with many variables) remains a challenge. The authors suggest the Expectation-Maximization (EM) algorithm as a potential path forward.
- Distinguishing Direct Influence: A major remaining hurdle is using these methods to distinguish between a common cause and a direct cause-effect relationship ($A \to B$) when only observations are available.
Summary updated