Abstract
The assumption of independence between observations (units) in a dataset is prevalent across various methodologies for learning causal graphical models. However, this assumption often finds itself in conflict with real-world data, posing challenges to accurate structure learning. We propose a decorrelation-based approach for causal graph learning on dependent binary data, where the local conditional distribution is defined by a latent utility model with dependent errors across units. We develop a pairwise maximum likelihood method to estimate the covariance matrix for the dependence among the units. Then, leveraging the estimated covariance matrix, we develop an EM-like iterative algorithm to generate and de-correlate samples of the latent utility variables, which serve as de-correlated data. Any standard causal discovery method can be applied on the de-correlated data to learn the underlying causal graph. We demonstrate that the proposed de-correlation approach significantly improves the accuracy in causal graph learning, through numerical experiments on both synthetic and real-world datasets.
Introduction
Most causal discovery methods, such as the PC algorithm or Greedy Equivalence Search (GES), rely heavily on the independent and identically distributed (i.i.d.) assumption. In many real-world scenarios involving binary data, this assumption is frequently violated. For instance, in single-cell RNA sequencing (scRNA-seq), cells may be related by lineage, and in social sciences, individuals within a group often share traits. When units are dependent, standard algorithms often misinterpret cross-unit correlations as variable-to-variable causal links, resulting in high false-positive rates. This paper introduces a decorrelation framework designed to map dependent binary data to a latent continuous space, estimate unit-wise covariance, and produce "clean" data suitable for standard causal discovery tools.
Methodology
The proposed approach transforms dependent binary data into independent continuous surrogates through a multi-step pipeline:
- Latent Utility Model: Binary variables ($X$) are modeled as thresholded versions of latent continuous variables ($Z$). While $Z$ follows a Directed Acyclic Graph (DAG) structure, its error terms ($\epsilon$) are correlated across units according to an unknown covariance matrix $\Sigma$.
- Pairwise Covariance Estimation: A pairwise Maximum Likelihood Estimation (MLE) method is developed to estimate the correlation between units from the binary latent data.
- EM-based Recovery: An Expectation-Maximization (EM) style algorithm is used. The E-step samples latent variables ($Z$) from a truncated normal distribution, while the M-step uses the estimated $\Sigma$ to decorrelate the data via a Cholesky transformation.
- Consensus Structure Learning: Standard causal discovery tools (e.g., MMHC, PC) are applied to the decorrelated data. A "Consensus" approach is employed, where discovery is run on multiple imputed datasets, retaining only edges that appear in at least 50% of the resulting graphs.
Experimental Results
The framework was evaluated on synthetic random DAGs and eight benchmark Bayesian networks from the bnlearn repository. The results indicate that capturing unit-wise dependency is critical for accurate structure learning.
On the benchmark networks, the Consensus approach yielded an average 20% improvement in F1-score compared to baselines that ignore unit dependency. In synthetic simulations with high dependency (equal covariance structures), F1-score improvements reached up to 34%. Notably, the method proved robust to model violations, performing well even when underlying causal relationships were non-linear.
Real-world Application
The method was applied to scRNA-seq data from Chu et al. (2016), involving 51 target genes across 1,018 cells. Cell-to-cell dependency is expected in this context due to shared progenitor populations.
The proposed model achieved a test-data log-likelihood ratio 1.57x higher than the baseline, confirming its ability to successfully capture biological dependencies between cells and provide a better fit for real-world data.
Conclusion
This research demonstrates that decorrelating dependent binary data significantly enhances the accuracy of causal discovery. While currently limited to binary data and assuming a block-diagonal covariance structure for computational feasibility, the framework provides a robust foundation for future extensions to multi-category and mixed data types. The computational cost remains a challenge for very large datasets, suggesting a need for further optimization in the pairwise MLE and iterative EM processes.
Summary updated