TL;DR
Standard causal discovery algorithms frequently fail when data is scarce, but incorporating expert or LLM knowledge as rigid constraints can lead to catastrophic errors if that knowledge is imperfect. The Guess2Graph (G2G) framework solves this by using unreliable expert "guesses" merely to optimize the sequence of statistical tests rather than replacing them. This approach guarantees statistical correctness even if the expert is entirely wrong, while significantly boosting accuracy when the expert is reasonably competent.
What Problem Does This Paper Address?
Causal discovery—the process of inferring a directed acyclic graph (DAG) of cause-and-effect relationships from observational data—is notoriously difficult when sample sizes are small. In these finite-sample regimes, statistical tests are error-prone, and because algorithms make decisions sequentially, a single early mistake can propagate and ruin the entire graph.
To mitigate this, researchers often turn to domain experts or Large Language Models (LLMs) to provide prior knowledge. However, traditional methods integrate this knowledge as either hard constraints (forcing the algorithm to accept or reject certain edges) or soft constraints (biasing the test statistics). The authors demonstrate that if the expert is wrong, hard constraints can cause unbounded error propagation, while soft constraints invalidate the underlying mathematical distributions of hypothesis tests.
The core challenge is: How can we safely leverage highly scalable but fallible experts (like LLMs) to improve finite-sample causal discovery without sacrificing worst-case statistical guarantees?
Methods
The authors introduce the Guess2Graph (G2G) framework. Instead of letting an expert override data-driven statistical tests, G2G uses the expert's predicted graph to reorder the sequence in which tests are executed.
The paper instantiates this framework in two constraint-based algorithms:
- PC-Guess: This directly augments the classic PC algorithm. It reorders the Edge Loop (prioritizing edges the expert thinks are false) and the Edge Prune subroutine (prioritizing conditioning sets the expert thinks will d-separate the variables).
- gPC-Guess (Guided PC): The authors realize that the standard PC algorithm is too rigid because it forces a level-by-level search (testing small conditioning sets before larger ones). gPC-Guess completely redesigns the PC algorithm into a single-pass approach, removing this "statistical conditioning bias" to make the algorithm fully responsive to expert guidance.
Datasets
The framework was evaluated using:
- Synthetic Data: Linear Gaussian structural equation models generated over sparse and dense Erdős-Rényi random graphs (specifically ER1 and ER3 topologies, with $d=20$ variables and $n=100$ samples).
- Real-World Data: The Sachs protein signaling dataset ($d=11$ variables, subsampled to $n=100$), which maps discovery against an established biological ground truth.
- Experts: Evaluated using both simulated experts with controlled accuracy levels ($p_\psi \in [0.3, 1.0]$) and a real LLM expert (Claude Opus 4.1).
Key Insights and Ideas
- The Asymmetry of Edge Decisions: The mathematical "aha moment" of the paper is that correctly removing a false edge early shrinks the adjacency sets for neighboring nodes, making subsequent true edges easier to find and retain. Conversely, retaining a true edge early does not alter adjacency sets or help remove false edges. Therefore, an algorithm should always try to test and eliminate false edges first.
- Algorithmic Rigidity Limits Guidance: Simply plugging expert knowledge into a rigid algorithm like PC yields only modest gains. Because PC insists on testing conditioning sets strictly by size (0, then 1, then 2...), it cannot immediately act on an expert's hint that a specific large conditioning set will eliminate a false edge. Redesigning the algorithm to allow immediate testing (as in gPC-Guess) is required to fully unlock the expert's value.
- Safety via Test Sequence Optimization: Because every edge decision is still ultimately validated by a statistical test, the algorithm achieves Statistical Consistency—meaning it will always converge to the true graph given infinite data, regardless of how terrible or adversarial the expert's guesses are.
Key Results
- Monotonic Improvement: For both algorithms, finite-sample performance (measured by skeleton F1 score) scales monotonically with expert accuracy.
- Redesign vs. Augmentation: In synthetic and real-world testing, PC-Guess showed modest improvements (up to 5%). In contrast, the redesigned gPC-Guess achieved up to a 30% performance gain when guided by an accurate expert.
- LLM Empowerment: When paired with Claude Opus 4.1 on the real-world Sachs dataset, gPC-Guess achieved a 15% performance boost over unguided baselines, outperforming both purely data-driven methods and the LLM's raw predictions alone.
- Robust Worst-Case Bounds: When tested with an entirely inaccurate expert ($p_\psi = 0$), the performance drop was strictly bounded to roughly 8 percentage points, proving the framework's safety-critical utility.
Limitations and Future Work
While the paper provides strong end-to-end finite-sample guarantees for constraint-based methods, extensions to score-based (e.g., SE-GES) and functional causal model-based (e.g., RESIT) methods are currently presented as conjectures rather than fully proven theorems. Additionally, the current framework assumes a static initial expert guess; the authors note that dynamically re-prompting an LLM as the algorithm confirms or denies edges is a promising but computationally expensive direction for future research.
Summary updated