Abstract
Traditional causal discovery algorithms often rely on the assumption of Independent and Identically Distributed (i.i.d.) data. However, in many real-world scenarios, such as social networks or biological tissues, observations are inherently dependent. This paper presents a novel framework for causal discovery on dependent binary data by recovering latent continuous variables through a decorrelation-based approach. By leveraging a latent utility model and an Expectation-Maximization (EM) algorithm, the proposed method allows standard causal discovery tools to achieve significantly higher accuracy on non-i.i.d. datasets.
Introduction: Beyond the I.I.D. Assumption
Causal discovery aims to uncover the underlying directed acyclic graph (DAG) that represents the causal relationships between variables in a system. A fundamental bottleneck in this field is the pervasive "i.i.d. habit"—the assumption that each data point is independent of the others. In practice, data is rarely independent. For instance, in single-cell RNA sequencing (scRNA-seq), cells from the same tissue sample share environmental and lineage-based dependencies. Similarly, in social science, individuals within the same community exhibit correlated behaviors.
When these dependencies are ignored, standard causal discovery algorithms frequently identify "ghost" edges—spurious correlations mistaken for causation—or fail to detect true causal links. This challenge is compounded when dealing with binary data (e.g., gene activation or binary survey responses), where the discrete nature of the data complicates the application of traditional continuous-data decorrelation techniques.
Methodology: The Decorrelation Framework
The authors, Alex Chen and Qing Zhou, propose a multi-stage pipeline designed to "cleanse" binary data of unit-level dependencies before applying causal discovery algorithms.
The Latent Utility Model
The core of the approach is treating observed binary variables ($X$) as thresholds of underlying, unobserved continuous variables ($Z$). This "latent utility" perspective posits that a binary variable flips to 1 only when its corresponding latent continuous value exceeds a certain threshold. Crucially, the authors prove in Lemma 2.1 that the causal structure of the observed binary variables is identical to that of these latent continuous variables, justifying a strategy focused on recovering $Z$.
Modeling and Estimating Dependence
The framework assumes that the noise in these latent variables is dependent across units, governed by an $n \times n$ covariance matrix ($\Sigma$). To handle the computational complexity of estimating such a large matrix, the researchers employ Pairwise Maximum Likelihood Estimation (MLE), which estimates correlations between units in pairs.
EM-based Decorrelation
The recovery of latent variables is achieved through an Expectation-Maximization (EM) algorithm. This iterative process alternates between an E-step, estimating the latent continuous values ($Z$), and an M-step, using the estimated covariance to "scrub" unit-level dependence via Cholesky factorization and updating the model parameters. Once the data is decorrelated and transformed into a continuous format, it can be processed by established causal discovery algorithms like the PC algorithm or Max-Min Hill Climbing (MMHC).
Experimental Validation
The researchers evaluated their framework using three distinct data categories: synthetic DAGs with various unit-dependence structures, eight "gold standard" benchmark Bayesian networks from the bnlearn repository (e.g., Hailfinder, Hepar2, and Pigs), and real-world scRNA-seq data from human embryonic stem cells.
Key Results and Insights
The proposed method demonstrated substantial improvements over baseline approaches that ignore unit dependence.
Accuracy Gains across Benchmarks
On average, the "Consensus" approach—which involves running the algorithm multiple times and selecting edges based on majority vote—improved the F1 score by approximately 20% compared to baseline methods. This improvement was consistent across various sample sizes and network complexities.
Biological Consistency
In the application to real-world scRNA-seq data, the decorrelation method achieved a 1.57x better likelihood ratio than the baseline. This indicates that the resulting causal graph is significantly more representative of the actual observed biological data, effectively accounting for cell-to-cell dependencies.
Robustness and Sparsity
The framework proved resilient even when the data-generating process included non-linear relationships. Furthermore, the researchers found that incorporating domain expertise—such as assuming a block-diagonal structure for the covariance matrix when group memberships are known—further enhances performance and scalability.
Discussion and Future Directions
While the framework marks a significant step forward, several areas remain for future exploration. Currently, the implementation is optimized for binary data; extending this to ordinal or mixed-type data is a logical next step. Additionally, the computational cost of EM iterations on very large datasets suggests a need for more efficient scaling strategies, perhaps through automated discovery of dependence clusters. By providing a robust "pre-processing" step for decorrelation, this work enables the application of sophisticated causal discovery tools to the complex, dependent datasets that characterize modern scientific research.
Summary updated