Abstract
Uncovering causal relationships in datasets that include both categorical and continuous variables is a challenging problem. The overwhelming majority of existing methods restrict their application to dealing with a single type of variable. Our contribution is a structural causal model designed to handle mixed-type data through a general function class. We present a theoretical foundation that specifies the conditions under which the directed acyclic graph underlying the causal model can be identified from observed data. In addition, we propose Mixed-type data Extension for Regression and Independence Testing (MERIT), enabling the discovery of causal connections in real-world classification settings. Our empirical studies demonstrate that MERIT outperforms its state-of-the-art competitor in causal discovery on relatively low-dimensional data.
Introduction
Causal discovery aims to identify directed relationships between variables from observational data. While many algorithms excel in purely continuous or purely discrete domains, real-world datasets are often "mixed-type," containing a combination of both. For instance, in clinical settings, a continuous drug dosage might influence a categorical patient outcome. Existing hybrid methods often rely on restrictive assumptions, such as Gaussianity, or fail to identify the full directed acyclic graph (DAG), instead returning a Markov Equivalence Class. This paper introduces the Mixed Additive Noise Model (MANM) and the MERIT algorithm to address these limitations, providing a framework for identifiable causal discovery in mixed-type settings.
Methodology
The authors propose a unified structural causal model, the Mixed Additive Noise Model (MANM), which handles variables based on their type:
- Continuous variables follow a standard additive noise model: $X = f(Parents) + Noise$.
- Categorical variables are represented via a $K$-level argmax model: $X = \text{argmax}_k (f_k(Parents) + Noise_k)$. This approach generalizes the multinomial logistic model, allowing for flexible noise distributions.
To learn the causal structure, the MERIT (Mixed-type data Extension for Regression and Independence Testing) algorithm is introduced. It operates in three distinct phases:
- Causal Ordering: The algorithm iteratively identifies leaf nodes. It checks if a continuous variable's residuals are independent of its predictors. If no continuous leaf is found, it searches for "categorical blocks" to remove.
- Continuous Pruning: Parents of continuous variables are identified using independence tests on residuals.
- Categorical Pruning: Parents of categorical variables are determined through conditional independence tests and functional constraints.
Empirical Evaluation
The performance of MERIT was evaluated using synthetic datasets (Erdös-Renyi and Scale-Free graphs) and real-world benchmarks, including the User Knowledge Modeling and Pima Indians Diabetes datasets.
As shown in the results table above, MERIT consistently outperforms state-of-the-art competitors such as HCM (Hybrid Causal Model) and GS (Generalized Score). Across various graph sizes ($p=3, 5, 10$), MERIT achieves lower Structural Hamming Distance (SHD) and Structural Intervention Distance (SID), while maintaining higher F1 scores. On real-world data, MERIT successfully identified 4 out of 5 known causal links in the User Knowledge dataset, whereas competitors often reversed causal directions.
Discussion and Conclusion
A key insight of this work is that the structural difference between additive noise models and argmax models makes the causal direction identifiable under mild conditions, without requiring specific noise distributions like Gaussian or Gumbel. By leveraging the "categorical block" concept, MERIT avoids common pitfalls in topological sorting for mixed graphs.
However, the algorithm faces a dimensionality bottleneck due to its reliance on joint independence tests (dHSIC), making it most effective for low-dimensional settings (10-20 variables). Future work will focus on scaling the algorithm to high-dimensional data and extending the model to account for unobserved confounders.
Summary updated