Scaling Causal Graph Discovery to High Dimensions

1,375 concepts1,030 papers9 proposals

Discovering cause-and-effect relationships from observational data is a foundational challenge in science and engineering. If we know the true causal graph—represented as a Directed Acyclic Graph (DAG)—we can predict the outcomes of interventions, design fair decision-making systems, and pinpoint the root causes of system failures.

However, traditional causal discovery algorithms hit a hard mathematical wall when applied to modern datasets. Whether dealing with thousands of genes, microservice metrics, or financial assets, the number of potential causal graphs scales super-exponentially with the number of variables. Overcoming this combinatorial explosion requires moving away from brute-force discrete searches toward continuous optimization, latent representations, and structural aggregation.

To understand why scaling causal discovery is difficult, consider the search space. For a system with $D$ variables, the number of possible DAGs grows at a rate of $2^{O(D^2)}$.

Historically, algorithms navigated this space using discrete search strategies (such as hill-climbing or evolutionary algorithms) or constraint-based conditional independence tests. As shown by Gang Li et al. in An Empirical Study of Encoding Schemes and Search Strategies in Discovering Causal Networks, these standard search strategies face severe scalability challenges due to this curse of dimensionality.

Even if we can reliably determine cause and effect between an isolated pair of variables—a task explored by Xiaohai Sun et al. in Distinguishing between cause and effect via kernel-based complexity measures for conditional distributions—scaling up to evaluate multivariate conditional distributions introduces immense statistical and computational complexity.

In real-world applications, this complexity manifests as a severe bottleneck:

To bypass this combinatorial wall, modern frameworks alter how the graph space is navigated and how variables are represented.

Differentiable Optimization and Continuous Relaxations

The most significant paradigm shift in high-dimensional causal discovery is continuous relaxation. Instead of treating graph discovery as a discrete search problem (where an edge is either present or absent), researchers frame it as a continuous optimization problem. Every potential edge is assigned a continuous weight, and the discrete requirement of acyclicity is formulated as a smooth, differentiable constraint function. This allows algorithms to use gradient descent to search the space of graphs.

To make this continuous optimization robust in high dimensions, Pingchuan Ma et al. developed a framework in Scalable Differentiable Causal Discovery in the Presence of Latent Confounders with Skeleton Posterior that uses a probabilistic skeleton posterior to guide the continuous optimization process. This prevents the algorithm from getting stuck in poor local minima when scaling to graphs with over a hundred variables.

Similarly, Ruichu Cai et al. demonstrated in Temporal latent variable structural causal model for causal discovery under external interferences that explicitly separating causal relationships into a binary adjacency matrix and a continuous weight matrix via variational inference helps filter out spurious correlations. This separation mitigates the statistical testing errors and autocorrelation issues that typically cause precision to degrade in high-dimensional time series.

Discrete Search (Exponential)       Continuous Relaxation (Differentiable)
   Graph A -> Graph B                  Weight Matrix W = [[0.0, 0.7],
   (Evaluate Score)                                       [0.0, 0.0]]
          ↓                                           ↓
   Hard Combinatorial Steps            Gradient Descent via Acyclicity Constraint

Causal Representation Learning in Latent Spaces

In many real-world scenarios, the true causal variables are not cleanly provided as tabular columns; instead, they are trapped inside high-dimensional, unstructured data like raw pixels. As Stefan Bauer et al. argued in Toward Causal Representation Learning, traditional causal discovery algorithms cannot scale to handle millions of raw parameters. Instead, models must jointly learn low-dimensional latent causal variables and their underlying Structural Causal Model (SCM).

Scaling these latent causal models is an active structural hurdle, as surveyed by Yongkai Wu et al. in From Identifiable Causal Representations to Controllable Counterfactual Generation: A Survey on Causal Generative Modeling. To address this, Mengyue Yang et al. embedded SCMs directly into Variational Autoencoders (VAEs) in CausalVAE: Disentangled Representation Learning via Neural Structural Causal Models, allowing the model to learn representations that respect real-world causal dependencies.

To avoid the computational overhead of explicit discrete graph optimization within these deep architectures, Johann Brehmer et al. introduced Implicit Latent Causal Models in Weakly supervised causal representation learning, leveraging weak supervision (such as temporal pairs before and after an intervention) to identify causal factors.

However, traditional point-estimate VAEs struggle to maintain accuracy as the dimensionality of the data grows. To scale further, Amir Mohammad Karimi Mamaghan et al. introduced a diffusion-based framework in Diffusion Based Causal Representation Learning. By utilizing infinite-dimensional latent trajectories rather than static point estimates, diffusion models preserve causal graph reconstruction accuracy far better in high-dimensional settings where traditional variational baselines degrade.

Exploiting Mathematical and Spatial Structure

Another powerful strategy for scaling is to exploit the intrinsic mathematical or physical structure of the data to simplify the optimization landscape.

Non-Gaussianity and Sparse Shocks

If we assume that the underlying drivers of a system—the structural shocks—are sparse and non-Gaussian, the causal directions become uniquely identifiable from observational data without requiring exhaustive conditional independence testing.

Markus Püschel demonstrated this in SpinSVAR: Estimating Structural Vector Autoregression Assuming Sparse Input by assuming that structural shocks are sparse and Laplacian-distributed. This mathematical assumption allows for a GPU-accelerated estimator that uncovers causal relationships in large-scale time series orders of magnitude faster than previous techniques, scaling efficiently to hundreds of nodes.

Spatial Aggregation

In spatiotemporal data (such as climate maps or brain scans), high dimensionality is often an artifact of high spatial resolution, meaning adjacent grid points contain highly redundant information.

In Discovering Latent Causal Graphs from Spatiotemporal Data, Kun Wang showed that using spatial kernels to aggregate redundant grid points into a low-dimensional latent space allows for the joint learning of the causal graph and latent time series. By compressing the spatial dimensions first, the algorithm makes previously intractable high-dimensional grids computationally manageable.

Divide, Conquer, and Aggregate

Rather than performing a single, massive global optimization over thousands of variables, scalability can also be achieved by breaking the problem down into smaller, localized subproblems and aggregating the results.

This is the core intuition behind the framework developed by Yujia Bao et al. in Sample, estimate, aggregate: A recipe for causal discovery foundation models. Their framework, SEA (Sample, Estimate, Aggregate), works by:

  1. Subsampling small subsets of variables from the high-dimensional dataset.
  2. Running classical, localized causal discovery algorithms on these subsets to act as noisy feature extractors.
  3. Aggregating the localized estimates across thousands of subsamples to form a robust global causal graph.

By shifting the burden from a single exponential global search to parallelizable local estimations, this divide-and-conquer approach bypasses traditional scaling bottlenecks and provides a viable path forward for building causal discovery foundation models.

History & citationVersion 2 · 2026-10-04

Current · Permanent link

  1. Version 2 (latest) · 2026-10-04 · 1030 papers
  2. Version 1 · 2026-10-04 · 815 papers
Cite this direction

Latest version (a living page):

@misc{lacuna_high_dimensional_causal_discovery_79ce1876,
  title = {high-dimensional causal discovery},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/high-dimensional-causal-discovery/rdr_79ce1876845645ab940c0b009c33c439},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-19},
}

This exact version (version 2, frozen text):

@misc{lacuna_high_dimensional_causal_discovery_79ce1876_v2,
  title = {high-dimensional causal discovery},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/high-dimensional-causal-discovery/rdr_79ce1876845645ab940c0b009c33c439/v/2},
  note = {Version 2, frozen 2026-10-04. Accessed 2026-11-19},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake