Causal discovery from Bayesian symbolic regression

Roger Guimerà, Kai-Hendrik Cohrs, Sergio Pedreira, Marta Sales-Pardo, Gustau Camps-VallsarXiv 2026 · arXiv

TL;DR

This paper reframes causal discovery as a Bayesian symbolic regression problem, using ensembles of mathematical expressions to identify causal "parents" of a variable. By calculating how frequently a candidate variable appears in a posterior distribution of plausible models, the authors create a robust causal score that outperforms specialized algorithms in challenging nonlinear, autocorrelated, and high-noise settings.

What Problem Does This Paper Address?

Identifying cause-and-effect relationships from observational data is a cornerstone of science, but it’s notoriously difficult. Traditional methods often hit a wall in two specific scenarios:

  1. Bivariate cases: If you only have two variables, $X$ and $Y$, standard statistical tests often can't tell if $X \to Y$ or $Y \to X$ because the marginal independence structure looks identical.
  2. Autocorrelation: In fields like climate science or neuroscience, variables are often strongly correlated with their own past values, which "muddies the water" for traditional causal discovery tools.

Furthermore, most existing tools only give you a graph (arrows between nodes). Scientists, however, often want the mechanism—the actual mathematical equation that describes how one variable influences another.

Methods

The authors introduce causalBMS, which leverages the Bayesian Machine Scientist (BMS). Instead of searching for a single "best" equation, BMS uses a Markov chain Monte Carlo (MCMC) sampler to explore the space of closed-form mathematical expressions (like $y = a \cdot \sin(bx) + c$).

The key innovation is the Posterior Inclusion Frequency (PIF). For a target variable $x_i$, the algorithm samples thousands of plausible equations. If a candidate parent variable $x_k$ appears in 90% of those equations, it receives a high causal score. This ensemble approach makes the method incredibly robust; even if the "perfect" equation isn't found, the consistent presence of a specific variable across many "good" equations strongly signals a causal link.

The method relies on the Minimum Description Length (MDL) principle, which mathematically balances the simplicity of an equation against how well it fits the data, preventing the model from overcomplicating the mechanism.

Datasets

The authors evaluated causalBMS against a battery of rigorous tests:

  • Coupled Nonlinear Bivariate Dynamics: A synthetic system specifically designed to be "hard" for linear and Granger-style methods due to strong autocorrelation.
  • CauseMe Benchmark Suite: A standard industry benchmark featuring linear, chaotic, and nonlinear datasets with varying dimensions ($d=3$ to $10$) and noise levels.
  • Biogeoscience Structural Causal Model (SCM): A four-variable model (Radiation, Temperature, Productivity, Respiration) that simulates real-world ecosystem dynamics, including non-Gaussian noise and "misspecified" time resolutions.

Key Insights and Ideas

  • The Learnability Transition: The paper connects causal discovery to a theoretical "learnability transition." There is a specific noise threshold ($\sigma^*$) beyond which the true mathematical model becomes unlearnable. Surprisingly, the authors found that causal structure can often be recovered even in the "unlearnable" high-noise regime.
  • Noise as a Signal: In some nonlinear systems, higher noise can actually increase the signal-to-noise ratio by pushing the system into a wider range of states, making the underlying functional form more visible to the symbolic regressor.
  • Beyond the Graph: Unlike "black-box" causal methods, causalBMS provides the explicit functional form of the interaction. This allows researchers to perform "residual adequacy" tests—checking if the discovered equation actually explains the data or if there's still a hidden signal left over.

![FIG.

  1. Causal and functional recovery in unidirectionally coupled nonlinear dynamics. The method correctly identifies causal links (e-h) and approximates the underlying equations (i-l) across four different noise regimes.](/figures/art_b45dacd59a4a466baae614f1f6c8df65)

Key Results

  • Bivariate Success: In the bivariate nonlinear test, causalBMS correctly identified the direction of causality ($X \to Y$) at all noise levels, whereas traditional Granger-style methods failed at moderate noise.
  • Benchmark Dominance: On the CauseMe leaderboard, causalBMS was a top performer in 13 out of 21 dataset-metric pairs, particularly excelling in nonlinear and high-noise variants where it consistently sat at the top of the competitive range.

![FIG.

  1. Performance on the CauseMe benchmark. Green bars show causalBMS consistently hitting the top of the range (grey) across AUC, AUC-PR, and F1-scores.](/figures/art_023a4e2e0c8d4e3f970da4190ccdf154)
  2. Interpretability in Earth Science: In the biogeoscience study, the method not only identified the causal links but, for the "Respiration" variable, it recovered the exact ground-truth mathematical expression used to generate the data.

Limitations and Future Work

  • Scalability: The MCMC sampling process is computationally expensive. While it handles 10 variables easily, it would likely struggle with hundreds of variables without a "pre-pruning" step to narrow down candidate parents.
  • Closed-form Assumption: The method assumes the true mechanism can be represented by a mathematical expression. If the reality is a "black box" that doesn't follow a closed-form rule, the consistency guarantees may not hold.
  • Future Directions: The authors suggest a hybrid approach: using fast, constraint-based methods to prune the search space (the "skeleton") and then using causalBMS to orient the edges and discover the specific mathematical mechanisms.

Summary updated

Abstract

Causal discovery from observational data is central to scientific reasoning, yet established constraint-based and score-based methods often recover only a Markov equivalence class and provide no signal in bivariate or strongly autocorrelated regimes typical in some areas of science. Here, we recast causal discovery as a Bayesian symbolic regression problem. Given a set of candidate parents for each target variable, the Bayesian symbolic regression samples closed-form expressions from a posterior over symbolic models. The fraction of sampled expressions in which each candidate variable appears defines a posterior inclusion probability that we use as the causal score. This replaces pointwise model selection by an ensemble of plausible mechanisms. We evaluate the method on coupled nonlinear bivariate dynamics, on the CauseMe benchmark suite spanning linear, chaotic, nonlinear, higher-dimensional and high-noise settings, and on a four-variable biogeoscience structural causal model under Gaussian, heavy-tailed and mixed-noise regimes. Our approach is competitive with, and often outperforms, specialized causal discovery algorithms. Additionally, the method returns closed-form mechanisms leading to explicit causality and interpretability.
BibTeX
@misc{guimera2026causal,
  author = {Roger Guimerà and Kai-Hendrik Cohrs and Sergio Pedreira and Marta Sales-Pardo and Gustau Camps-Valls},
  title = {{Causal discovery from Bayesian symbolic regression}},
  year = {2026},
  eprint = {2611.12458},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.12458},
}
Figure