Composable Causality: A Toolkit for Systematic Time-Series Causal Discovery and Treatment-Effect Benchmarking

Muhammad Hasan Ferdous, Md Osman GaniarXiv 2026 · arXiv

Evaluating causal relationships in time-series data is notoriously difficult. Unlike standard machine learning, where we can just look at prediction accuracy, causal discovery requires knowing the "ground truth" of how variables influence each other over time—something rarely available in real-world datasets.

A new paper from the Causal AI Lab at UMBC introduces DTCD (Datasets and Toolkit for Causal Discovery), a modular framework designed to bring scientific rigor to this field. By moving away from static, "one-size-fits-all" datasets, DTCD allows researchers to stress-test algorithms against specific challenges like non-linearity, missing data, and non-stationarity in a controlled, composable way.

TL;DR

DTCD is a configurable toolkit for generating synthetic time-series data with exact ground-truth causality and counterfactual outcomes. It enables researchers to isolate specific algorithm failure modes—such as sensitivity to missing data or "cheating" via marginal variance shortcuts—that are typically hidden by aggregate benchmark scores.

What Problem Does This Paper Address?

Most researchers in time-series causal discovery rely on synthetic data because real-world ground truth is elusive. However, existing benchmarks suffer from two major flaws:

  1. Rigidity: They are often static files or hardcoded scripts. If you want to see how an algorithm handles only increased sparsity while keeping the noise distribution identical, you usually have to write new generator code from scratch.
  2. Evaluation Gaps: Most benchmarks only provide observational data. Evaluating "Individual Treatment Effects" (ITE) requires knowing what would have happened if a variable were different (a counterfactual). Existing tools rarely provide these exact "twins."

Furthermore, many benchmarks are "easy to game." Algorithms can sometimes achieve high scores not by finding causal links, but by exploiting statistical shortcuts like varsortability—where variables are simply sorted by their marginal variance.

Methods

The core of DTCD is its compositional architecture. As shown in Figure 1, the data-generating process (DGP) is broken down into five independent, orthogonal axes:

Figure 1: Compositional structure of DTCD. Five independent configuration axes feed one structural rollout, which emits the trajectory, the ground-truth adjacencies, the observation mask, and, when an intervention policy is supplied, an exactly paired counterfactual twin via re-injection of the cached noise N. Varying one axis holds all others fixed by construction.
  • Topology: The "skeleton" of the graph (e.g., Erdős-Rényi, Scale-free).
  • Mechanism: The functional form of relationships (Linear, Polynomial, MLP, etc.).
  • Noise: The distribution of stochastic innovations (Gaussian, Laplace, Cauchy, etc.).
  • Mask: How data goes missing (MCAR - random, or MNAR - value-dependent).
  • Intervention: Policies for "do-calculus" style changes to the system.

A standout feature is the Counterfactual Suite. By caching the exact noise realizations during a "factual" run and re-injecting them during an "interventional" run, DTCD produces counterfactual twins that are exact to floating-point precision. This allows for the first truly precise measurement of ITE in a temporal benchmark.

Datasets

The authors introduce the DTCD library and a reference benchmark consisting of:

  • 9 Graph Templates: Varying in dimensionality ($d \in \{4, 6, 8\}$) and maximum lag ($\tau_{max} \in \{2, 3, 4\}$).
  • 81 Discovery Configurations: Testing combinations of linear/nonlinear mechanisms, drift, and missingness.
  • 18 CATE Configurations: For treatment-effect estimation.

They compare eight discovery algorithms (like PCMCI, DYNOTEARS, and VAR-LiNGAM) and four treatment-effect estimators (like TARNet and X-Learner).

Key Insights and Ideas

  • The "Varsortability" Audit: The authors found that their raw benchmark had a varsortability of 0.93 (where 1.0 is perfectly sortable by variance). By adding a "standardization" toggle, they forced algorithms to actually learn structure. When standardized, the top-performing algorithm's (DYNOTEARS) F1 score dropped from 0.99 to 0.82, proving it was partially relying on the variance shortcut.
  • Missing Data is a Hard Wall: Most discovery methods (6 out of 8) simply cannot handle NaN values and fail immediately. Even for those that do, like CUTS+, there is a significant performance gap between random missingness (MCAR) and informative missingness (MNAR).
  • Aggregate Metrics Hide Nuance: A method might have a low True Positive Rate (TPR) not because it's "bad," but because of a scope mismatch. For example, CD-NOD is designed for contemporaneous edges and ignores lagged ones; DTCD's granular reporting makes this clear rather than just giving it a "failing" grade.

Key Results

The benchmark results (summarized in Figures 2 and 3) highlight clear winners and specific vulnerabilities:

  • DYNOTEARS dominates clean, linear conditions with an F1 of ~0.99 but is the most affected by the varsortability shortcut.
  • CUTS+ is the clear leader for missing data, outperforming mean-imputation strategies on both MCAR and MNAR variants.
  • TARNet proved to be the most efficient CATE estimator, winning on both linear and nonlinear variants while being significantly faster than more complex architectures like SNet.
Table 1: Causal discovery performance on Linear and Nonlinear DGP variants. Mean (± std) over 90 runs per cell (9 graphs × 10 seeds), filtered to status=ok rows. Lower SHD and FDR are better; higher TPR is better. Bold marks the best value in each column. PCMCI-GPDC values reflect d=4 cells only (n=30 versus n=90 for other methods). Performance on Drift is within 0.08 SHD of Linear for every method; full results in Appendix H.
Table 2: Causal discovery performance on missingness DGP variants. Only the two methods declaring supports_missing=True are shown; the remaining six methods were skipped on these variants (see Appendix I). Bold marks the best value in each column.

Limitations and Future Work

While powerful, DTCD is still a synthetic generator. The authors acknowledge a synthetic-to-real gap; high performance on DTCD is a prerequisite for, but not a guarantee of, success on real-world data.

Currently, the reference benchmark is limited to relatively small graphs ($d=8$), though the toolkit supports larger ones. Future work includes adding machine-readable identifiability tags to warn users when a configuration is theoretically impossible to solve, and expanding the CATE benchmark to include more recent architectures like Causal Transformers.

For engineers and researchers, DTCD provides a much-needed "diagnostic lab" to understand exactly where their causal models break. The code is open-source and available at github.com/hferdous/Composable_Causality.

Summary updated

Abstract

Real-world multivariate time series rarely come with verified ground-truth causal structure, so research on time-series causal discovery and treatment-effect estimation depends on synthetic data. Existing tools are either fixed datasets that cannot be modified after release or script-based generators in which each script fixes one combination of mechanism, confounding, and missingness. Neither supports controlled experiments that vary a single property, such as sparsity, noise distribution, or missingness mechanism, while holding the others fixed. Counterfactual evaluation has the same gap, since existing simulators are built per paper with hardcoded mechanisms or produce observational data only. We introduce DTCD (Datasets and Toolkit for Causal Discovery), a data generation library that produces multivariate time series with ground-truth contemporaneous and lagged adjacency matrices. Topology, mechanism, noise, observational mask, and intervention policy are independent components that compose through configuration alone. Paired counterfactual twins, produced by caching the stochastic realization during the factual rollout and re-injecting it during the interventional rollout, recover individual treatment effects to numerical precision. A reference benchmark of eight discovery algorithms on eighty-one configurations and four treatment-effect estimators on eighteen configurations exposes failure modes that aggregate metrics hide. These include the inability of standard discovery methods to handle missing values, regardless of whether the missingness mechanism is informative, and a marginal-variance shortcut (varsortability) that we measure and remove by standardization. Code and data: https://github.com/hferdous/Composable_Causality
BibTeX
@misc{ferdous2026composable,
  author = {Muhammad Hasan Ferdous and Md Osman Gani},
  title = {{Composable Causality: A Toolkit for Systematic Time-Series Causal Discovery and Treatment-Effect Benchmarking}},
  year = {2026},
  eprint = {2611.08918},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.08918},
}
Figure