Most foundational algorithms for causal discovery—the task of learning cause-and-effect graphs from observational data—operate under a convenient but unrealistic assumption: all variables in a system share the same data type. They assume the world is either entirely continuous (allowing for linear regressions and differential equations) or entirely discrete (allowing for conditional probability tables and multinomial distributions).
Real-world data is rarely so uniform. In clinical settings, a patient's record combines continuous measurements like blood pressure with binary indicators like treatment groups and categorical classifications like genetic variants. Forcing these heterogeneous data types into a single format creates a dilemma. Discretizing continuous variables throws away granular information and introduces structural errors, while treating categorical labels as continuous numbers violates basic statistical assumptions, leading to spurious causal links.
To solve this, researchers have developed mixed-type causal discovery, a paradigm that natively models the interactions between diverse data distributions. Understanding how it works requires looking at how heterogeneity changes the two main pillars of causal discovery: constraint-based conditional independence testing and score-based structural modeling.
The Statistical Hurdle: Testing Independence Across Types
Constraint-based causal discovery algorithms, such as the PC or FCI algorithms, reconstruct causal graphs by systematically testing for conditional independence ($X \perp Y \mid Z$). If two variables remain dependent regardless of what you condition on, a causal edge exists between them.
However, computing a $p$-value or a test statistic becomes exceptionally difficult when $X$, $Y$, and $Z$ are a mix of continuous and discrete variables. For instance, how do you condition a continuous variable on a mixture of categorical error codes and real-valued temperatures?
Empirical evaluations by Raghu et al. demonstrated that in mixed-type settings, the choice of the underlying conditional independence test matters far more than the graph search algorithm itself. To bridge this gap, early constraint-based approaches relied on parametric frameworks like Conditional Gaussian (CG) networks, which assume that continuous variables follow a Gaussian distribution given any assignment of the discrete variables. As noted by Tsagris et al., integrating specialized tests like multinomial logistic regression directly into constraint-based frameworks allows algorithms to handle mixed data without altering the underlying search logic.
When parametric assumptions fail, researchers turn to information theory. If we can accurately estimate Conditional Mutual Information (CMI) for mixed distributions, we can use it as a universal metric for independence. The challenge is that standard entropy estimators fail when continuous and discrete components mix.
To resolve this, Gao et al. developed a non-parametric mutual information estimator that calculates the Radon-Nikodym derivative using $k$-nearest neighbor distances, bypassing the need to calculate separate continuous and discrete entropies. Similarly, Moon et al. introduced GENIE, an ensemble estimator that achieves optimal parametric convergence rates on mixed data, making it highly effective for high-dimensional biological datasets.
For highly irregular distributions—such as zero-inflated rainfall data that acts as a discrete zero but a continuous value otherwise—Marx et al. demonstrated that multi-dimensional adaptive histograms can dynamically bin data to preserve true causal links while eliminating the persistent biases that plague kernel-based estimators.
Hybrid Structural Equations: Exploiting Asymmetry
While constraint-based methods use independence tests to find where edges belong, score-based and functional causal models focus on determining the direction of the causal arrow (e.g., distinguishing $X \to Y$ from $Y \to X$).
In purely continuous systems, frameworks like Additive Noise Models (ANMs) or LiNGAM exploit the fact that if $X \to Y$ is a linear or non-linear function with independent noise ($Y = f(X) + \epsilon$), regressing $X$ on $Y$ will yield a residual that is dependent on $Y$. This mathematical asymmetry breaks the causal symmetry.
In mixed-type systems, this asymmetry is even more pronounced because the functional form changes depending on the direction of the edge. Consider a binary variable $B$ and a continuous variable $C$:
- If $B \to C$, the relationship is naturally modeled as an additive noise framework where the distribution of $C$ shifts based on the state of $B$:
$$C = \alpha B + \epsilon$$ - If $C \to B$, a continuous variable is causing a discrete state. It makes no sense to add continuous noise to a binary choice. Instead, the relationship must be modeled via a conditional probability or a classification function, such as a logistic link:
$$P(B=1 \mid C) = \sigma(\gamma C)$$
Zeng et al. and Li et al. formalized this insight into Hybrid Structural Equation Models (SEMs). By explicitly modeling continuous nodes using non-linear additive noise models and discrete nodes through classification-based formulations, the authors proved that the true causal direction between mixed pairs becomes fully identifiable from observational data.
Causal Direction Asymmetry:
Discrete (B) ----> Continuous (C) Continuous (C) ----> Discrete (B)
C = f(B) + Noise (ANM) P(B|C) = Logistic/Softmax
Instead of treating data heterogeneity as a hurdle, these hybrid frameworks treat it as an asset: the structural mismatch between data types provides strong mathematical clues about which variable is the cause and which is the effect.
Universal Metrics via Information Complexity
Another way to unify mixed data types without explicitly defining hybrid equations for every possible pair of distributions is to use the Minimum Description Length (MDL) principle. MDL views causal discovery through the lens of data compression: the true causal graph is the one that allows for the shortest joint description of the data and the model.
Marx et al. leveraged this concept to create a universal causal discovery framework. Because the algorithmic complexity of a dataset does not depend on whether a variable is a real number or a categorical label, MDL provides a level playing field. If $X$ causes $Y$, then describing the mechanism of $X$ and then describing $Y$ given $X$ requires fewer bits than describing $Y$ first and explaining $X$ given $Y$. This information-theoretic approach sidesteps the need to manually engineer specific parametric models for every new mixture of variables.
Real-World Applications and Benchmarks
The push toward mixed-type causal discovery is heavily driven by practical limitations in high-stakes domains. Historically, as Blum noted in early clinical AI research, prioritizing real-valued variables over binary or rank-valued data severely limits the utility of automated discovery in medicine.
Modern applications show why handling mixed types natively is crucial:
- Genomics and Multi-Omics: Abécassis et al. demonstrated that identifying the direct causal drivers of cancer drug responses requires integrating discrete genetic variants (SNPs) with continuous gene expression data. Standard tools fail to separate true mechanisms from simple correlations in these mixed-modal environments.
- Clinical Risk Prediction: In predicting hospital-acquired injuries, frameworks developed by Li et al. combine categorical medical histories with continuous physiological metrics, providing clinicians with transparent, explainable causal roadmaps rather than black-box correlations.
- Industrial Benchmarking: To evaluate how these algorithms perform outside of idealized simulations, Göbler et al. introduced
causalAssembly, a benchmark that mimics real-world manufacturing lines by blending continuous sensor streams with discrete error codes and batch IDs.
By moving away from uniform data assumptions and embracing the mathematical asymmetries inherent in mixed data, modern causal discovery algorithms can faithfully map the hidden networks that drive complex, heterogeneous systems.