Causal Discovery from Overlapping Variable Sets

This direction is merged into another direction: causal data fusion. Its last version stays readable and citable here.

Version 1 of 113 papers18 conceptsfrozen 2026-10-04

In an ideal world, scientific discovery would happen on a single, massive spreadsheet containing every variable of interest for every subject. In reality, data is fragmented. One hospital might record a patient's genetics and symptoms, while another records their symptoms and treatment outcomes. Because the genetics and outcomes are never measured in the same room, standard machine learning tools—which require a unified data matrix—cannot tell us if the genes cause the outcome.

Researchers like Asimakis P. Mariglis and Sofia Triantafillou have developed a framework to solve this "data silo" problem. By leveraging the logic of causality, we can reconstruct a global picture of a system from overlapping, partial observations.

The Intuition of the Bridge Variable

The core mechanism behind this research is the use of "bridge" variables. Suppose we have two datasets: Dataset A contains variables $\{X, Z\}$ and Dataset B contains $\{Z, Y\}$. We want to know the relationship between $X$ and $Y$, even though they were never co-measured.

The variable $Z$ acts as an anchor. If we can determine the causal relationship between $X$ and $Z$ in the first study, and the relationship between $Z$ and $Y$ in the second, we can use the rules of Directed Acyclic Graphs (DAGs) to constrain the possible relationships between $X$ and $Y$. As Eva Endres and Thomas Augustin demonstrate in Statistical Matching of Discrete Data by Bayesian Networks, Bayesian Networks allow us to represent these conditional independence assumptions to estimate joint distributions that were never directly observed.

From Merging Graphs to Global Estimation

Early attempts at this problem focused on "post-hoc merging." Researchers would build a small causal graph for each dataset and then try to stitch them together. However, this often led to logical conflicts or a loss of information.

A more robust approach, championed by Biwei Huang and Kun Zhang, involves simultaneous global estimation. In Causal Discovery from Multiple Data Sets with Non-Identical Variable Sets, they introduce CD-MiNi, which estimates a global causal matrix across all available datasets at once.

This works because different datasets provide different "views" of the same underlying causal mechanism. By assuming the causal structure is invariant across studies, we can use the unique statistical signatures in one dataset—such as non-Gaussian noise—to resolve ambiguities in another. This allows for the reconstruction of a complete DAG even when no single source contains the full state space of the system.

Solving the Search Problem with Constraints

As the number of variables grows, the number of possible causal graphs explodes exponentially. Integrating these overlapping networks becomes a massive combinatorial puzzle. Tom Claassen and Tom Heskes showed that we can treat causal discovery as a search for a "truth" that is consistent across all experimental constraints.

To make this search efficient, Praveen Nair and Sergey Plis introduced ION-C (Integration of Overlapping Networks via Constraints). In ION-C: Integration of Overlapping Networks via Constraints, they use Answer Set Programming (ASP)—a form of declarative logic programming—to encode causal rules as hard constraints. This allows the algorithm to quickly prune the space of impossible graphs, scaling the discovery process from small toy examples to complex networks with dozens of variables.

Measuring Hidden Effects

This research direction isn't just about drawing arrows between variables; it has practical implications for policy and medicine. Kazuhiko Shinoda and Takahiro Hoshino have applied these concepts to estimate the Local Average Treatment Effect (LATE).

In Estimation of Local Average Treatment Effect by Data Combination, they address scenarios where treatment assignments are in one database and outcomes are in another. By using overlapping covariates to "link" these unlinked databases, they can measure the effectiveness of an intervention without needing a single, privacy-compromising master list of individuals. This proves that causal logic can act as a bridge across the gaps created by data fragmentation and privacy laws.

Papers in this version

History & citationVersion 1 · 2026-10-04

Merged into another direction · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_causal_discovery_from_overlapping_variab_1c92984f,
  title = {Causal Discovery from Overlapping Variable Sets},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/causal-discovery-from-overlapping-variable-sets/rdr_1c92984f794941018070d295b876422a},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-22},
}

This exact version (version 1, frozen text):

@misc{lacuna_causal_discovery_from_overlapping_variab_1c92984f_v1,
  title = {Causal Discovery from Overlapping Variable Sets},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/causal-discovery-from-overlapping-variable-sets/rdr_1c92984f794941018070d295b876422a/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-22},
}