The most likely common cause

A. A. Hovhannisyan, A. E. Allahverdyan

Abstract

The common cause principle for two random variables $A$ and $B$ is examined in the case of causal insufficiency, when their common cause $C$ is known to exist, but only the joint probability of $A$ and $B$ is observed. As a result, $C$ cannot be uniquely identified (the latent confounder problem). We show that the generalized maximum likelihood method can be applied to this situation and allows identification of $C$ that is consistent with the common cause principle. It closely relates to the maximum entropy principle. Investigation of the two binary symmetric variables reveals a non-analytic behavior of conditional probabilities reminiscent of a second-order phase transition. This occurs during the transition from correlation to anti-correlation in the observed probability distribution. The relation between the generalized likelihood approach and alternative methods, such as predictive likelihood and the minimum common cause entropy, is discussed. The consideration of the common cause for three observed variables (and one hidden cause) uncovers causal structures that defy representation through directed acyclic graphs with the Markov condition.

Introduction: The Latent Confounder Problem

In causal inference, the Common Cause Principle (CCP) posits that if two events are correlated without one causing the other, they must share a common cause. However, researchers often face causal insufficiency: we know a common cause $C$ exists for variables $A$ and $B$, but we can only observe the joint probability $p(a, b)$. Because $C$ is hidden (latent), the system is non-identifiable; infinitely many configurations of $C$ can mathematically satisfy the observed data.

Hovhannisyan and Allahverdyan address this by proposing a Generalized Maximum Likelihood (GL) framework. By borrowing concepts from statistical mechanics and information theory, they provide a principled method to reconstruct the "most likely" latent cause, uncovering surprising physical-like behaviors in causal structures.

Methodology: Generalized Maximum Likelihood

The authors utilize a likelihood function $L_\beta(\theta)$ parameterized by a hyperparameter $\beta$. This approach bridges causal inference with the Maximum Entropy principle. Specifically, when $\beta \lesssim 1$, maximizing the likelihood is equivalent to maximizing the conditional entropy $H(C|AB)$. This ensures that the identified cause $C$ introduces the minimum amount of "extra" information not already present in the observed data.

For complex scenarios involving three or more observed variables, the parameters of the common cause are found via numerical optimization under the constraints of the observed joint distribution. This method naturally favors causes that act as "better predictors" of the effects than the effects are of each other—a key desideratum for a true common cause.

Causal Phase Transitions

A primary discovery of this work is that the "most likely" cause does not always evolve smoothly as data changes. In the case of binary symmetric variables, as the observed distribution shifts from correlation (where $A$ and $B$ occur together) to anti-correlation, the conditional probabilities of the common cause exhibit non-analytic behavior.

Conditional probabilities of the common cause exhibiting non-analytic behavior (phase transitions) as the observed joint distribution varies.

As seen in Figure 1, while the first derivatives of the probabilities remain continuous, the sudden change in behavior at specific thresholds is reminiscent of a second-order phase transition in physics. This suggests that causal structures can undergo discrete "regime shifts" based on the strength and nature of observed correlations.

Beyond Directed Acyclic Graphs

The study of three observed variables ($A_1, A_2, B$) reveals further complexity. The "most likely" common cause $C$ can result in dependencies that standard Directed Acyclic Graphs (DAGs)—the workhorse of modern causal modeling—struggle to represent under the Markov condition.

Causal structures for three observed variables where the latent cause C induces dependencies that challenge standard DAG representations.

This "DAG defiance" implies that the underlying causal reality might be more interconnected than sparse graphical models suggest. The GL method avoids the pitfalls of "sparsity-seeking" alternatives like Minimum Common Entropy, which often force probabilities to zero or one as a mathematical artifact rather than a physical reality.

Key Results and Implications

  • Consistency with Independence: If $A$ and $B$ are independent, the GL method correctly identifies a uniform distribution for $C$, indicating no predictive common cause exists.
  • Geyser Model: Applying the method to Reichenbach’s classic "geyser" example, the GL approach identifies a rare, quasi-deterministic common cause ($p(c) \approx 0.11$) that triggers both effects with high probability ($p(a|c) = p(b|c) \approx 0.94$).
  • Predictive Power: Unlike alternative methods that may produce causes with less predictive power than the variables themselves, the GL method ensures the common cause is a superior predictor of its effects.

Conclusion and Future Directions

While the GL framework provides a robust path for identifying latent confounders, several challenges remain. Future research must address the automatic determination of the dimensionality of $C$ and scale the method to continuous or high-dimensional variables. Additionally, distinguishing between a common cause and a direct $A \to B$ relationship remains a fundamental hurdle in purely observational settings.

Summary updated

Abstract

The common cause principle for two random variables $A$ and $B$ is examined in the case of causal insufficiency, when their common cause $C$ is known to exist, but only the joint probability of $A$ and $B$ is observed. As a result, $C$ cannot be uniquely identified (the latent confounder problem). We show that the generalized maximum likelihood method can be applied to this situation and allows identification of $C$ that is consistent with the common cause principle. It closely relates to the maximum entropy principle. Investigation of the two binary symmetric variables reveals a non-analytic behavior of conditional probabilities reminiscent of a second-order phase transition. This occurs during the transition from correlation to anti-correlation in the observed probability distribution. The relation between the generalized likelihood approach and alternative methods, such as predictive likelihood and the minimum common cause entropy, is discussed. The consideration of the common cause for three observed variables (and one hidden cause) uncovers causal structures that defy representation through directed acyclic graphs with the Markov condition.
BibTeX
@article{hovhannisyan2024most,
  author = {A. A. Hovhannisyan and A. E. Allahverdyan},
  title = {{The most likely common cause}},
  journal = {International Journal of Approximate Reasoning},
  year = {2024},
  doi = {10.1016/j.ijar.2024.109264},
}
Figure