This document is a formal discussion of a research paper by Tillman and Spirtes (T&S) regarding causal discovery. As a "discussion paper," it provides a critical analysis, contextualization, and extension of the original work rather than presenting new experimental data.
TL;DR
This paper discusses a novel framework for "stitching together" causal knowledge from multiple datasets that measure different, overlapping sets of variables. It highlights how using multiple distributions can actually make causal discovery more robust against statistical errors like "unfaithfulness," while critiquing the realism of assuming identical selection bias across different studies.
What Problem Does This Paper Address?
In an ideal world, causal discovery—the process of finding cause-and-effect relationships from data—would happen using one massive dataset containing every variable of interest. In reality, researchers usually have access to several smaller studies, each measuring only a subset of variables (e.g., Study A measures $\{X, Y\}$, Study B measures $\{Y, Z\}$).
The challenge is how to learn a single, coherent causal structure (a Directed Acyclic Graph or DAG) when no single dataset contains all the variables. Standard algorithms rely on an "oracle" that can answer whether any two variables are independent. When variables are never measured together, the oracle is "incomplete." The T&S paper provides a constraint-based algorithm to navigate these gaps, even when hidden (latent) variables or selection biases are present.
Methods
The discussion focuses on the constraint-based approach used by T&S. Key methodological points include:
- Constraint-Based Learning: Instead of looking for the "best" model, the algorithm eliminates models that are incompatible with the observed conditional independencies in the data.
- Handling Incomplete Oracles: The algorithm is designed to work even when certain independence queries cannot be answered because the variables were never jointly recorded.
- Meta-Analysis Integration: T&S use Fisher’s method to combine p-values from independence tests across different datasets.
- Common Structure, Different Parameters: The method assumes all datasets share the same qualitative causal graph but allows them to have different probability distributions (different "strengths" of effects).
Datasets
As this is a theoretical discussion paper, it does not introduce new datasets or benchmarks. It references the general scenario of overlapping variables common in fields like medicine, psychology, and social sciences (e.g., different surveys asking different but related questions about mental health).
Key Insights and Ideas
- The "Faithfulness" Advantage: One of the most striking insights is that having multiple datasets with different distributions is a major benefit. In causal discovery, a "faithfulness violation" occurs when an accidental cancellation of parameters makes two variables look independent when they are actually causally linked. The discussants argue that while a violation might happen in one dataset, it is highly unlikely to happen across all of them, making the multi-dataset approach more robust.
- Selection Bias Skepticism: The discussants (Zhang and Silva) raise a red flag regarding T&S's assumption of "common selection bias." They argue it is unlikely that different studies, conducted by different researchers at different times, would suffer from the exact same selection mechanisms. They suggest this is a primary area for future relaxation of assumptions.
- Latent Variable Inference: The framework can be used to infer the existence of latent (unobserved) variables. For example, if multiple observed variables are correlated in specific ways across different datasets, one can mathematically "unveil" the hidden common cause driving them.
Key Results
The "results" of this discussion are primarily conceptual and theoretical:
- Validation of the T&S Framework: The discussants conclude that T&S’s approach is a promising solution for learning from incomplete information.
- Path to Score-Based Methods: The authors suggest that while T&S used a constraint-based method, the field could move toward score-based methods (like Likelihood or Bayesian approaches) by using "penalized composite likelihood" to handle the non-identical distributions across datasets.
Limitations and Future Work
- Relaxing Selection Bias: Future research needs to address what happens when selection bias differs across studies or when researchers intentionally introduce bias through experimental design.
- Non-Identical Structures: The current framework assumes the underlying causal graph is identical across all datasets. A natural next step is handling cases where some datasets come from observational studies and others from controlled experiments (interventions), where the graph structure actually changes.
- Computational Complexity: Moving from constraint-based algorithms to full model-based or Bayesian approaches for overlapping datasets is computationally expensive and requires new optimization strategies.
Summary updated