TL;DR
This paper introduces MissNODAG, a novel framework for learning causal relationships in complex systems that exhibit both feedback loops (cycles) and missing data, even when the missingness is not random. By combining an expectation-maximization approach with differentiable learning, MissNODAG can simultaneously uncover the underlying cyclic causal graph and the mechanism causing data to be missing, offering a more robust solution for real-world data.
What Problem Does This Paper Address?
Causal discovery, the process of identifying cause-and-effect relationships, is crucial for understanding complex systems like biological networks. However, existing methods often make simplifying assumptions: either the data is complete and the causal relationships form a Directed Acyclic Graph (DAG), or they struggle with data that is missing not at random (MNAR). Real-world systems frequently violate these assumptions, featuring cyclic dependencies and incomplete data where missingness patterns are informative. This gap limits the applicability of current causal discovery techniques. MissNODAG aims to bridge this gap by handling both cyclic causal structures and informative missing data simultaneously.
Methods
MissNODAG employs a differentiable framework that integrates an additive noise model with an expectation-maximization (EM) algorithm. The core idea is to iteratively:
- Impute missing values: Using a rejection sampling strategy, MissNODAG estimates the missing data points. This step is crucial for handling MNAR data.
- Optimize the causal graph and missingness mechanism: The framework jointly learns the parameters of the cyclic causal graph (using a differentiable approach inspired by NOTEARS) and the parameters governing the missingness mechanism. This is achieved by maximizing the expected log-likelihood of the observed data.
Key novelties include:
- Handling cyclic graphs: Unlike many methods that assume acyclicity, MissNODAG explicitly models and learns cyclic relationships.
- MNAR data handling: The EM framework, coupled with rejection sampling, allows for the modeling of missingness mechanisms that depend on the observed variables themselves, going beyond simpler MAR assumptions.
- Joint learning: The framework simultaneously learns the causal structure and the missingness mechanism, recognizing their interdependence.
- Differentiable approach: This allows for efficient optimization using gradient-based methods.
Datasets
The authors evaluated MissNODAG on:
- Synthetic datasets: Generated cyclic directed graphs with varying degrees of nonlinearity, cycles, missing probabilities, and numbers of interventions.
- Real-world gene perturbation data: Specifically, the Perturb-CITE-seq dataset, which allows for studying causal relations in gene networks at scale. They also used the Sachs et al. (2005) protein signaling dataset as a benchmark.
Key Insights and Ideas
- Cycles and missingness are intertwined: The paper highlights that ignoring cyclic structures or assuming simple missingness mechanisms can lead to inaccurate causal discovery. MissNODAG's ability to jointly learn both is a significant advantage.
- MNAR is common and important: The success on real-world data, particularly the scRNA-seq dataset where dropouts (a form of MNAR) are prevalent, underscores the importance of accounting for informative missingness.
- Differentiable EM is powerful: The combination of differentiable learning for the causal graph and an EM approach for imputation and missingness mechanism learning provides a scalable and effective solution.
- Robustness to missingness: MissNODAG demonstrates strong performance even with high missing probabilities, outperforming baselines that either assume complete data or simpler missingness models.
Key Results
- Target Law Recovery: MissNODAG consistently outperforms or matches baseline methods across various synthetic settings, achieving near-perfect recovery up to a missing probability of 0.3. It also shows strong scalability, remaining the top-performing model for graphs up to 40 nodes.
- Missingness Mechanism Recovery: MissNODAG consistently outperforms baselines in recovering the missingness mechanism (X→R and R→R edges), demonstrating its ability to learn complex missingness patterns.
- Real-World Performance: On the Perturb-CITE-seq dataset, MissNODAG outperforms baseline methods in terms of interventional negative log-likelihood (I-NLL), especially when treating zero gene expression values as missing. It also successfully recovers known regulatory structures and feedback loops in the gene network.
- Computational Cost: While MissNODAG is computationally more intensive than some baselines due to its joint learning and imputation steps, its performance gains justify the added cost.
Limitations and Future Work
- Identifiability Assumptions: The theoretical guarantees rely on strict identifiability assumptions for the missingness mechanism. Violating these can degrade performance.
- Non-convex Optimization: The EM procedure guarantees convergence to a stationary point, not necessarily a global maximum, due to the non-convex optimization landscape.
- Scalability: The log-determinant computation for the Jacobian imposes a complexity of O(K³L), limiting scalability to very large graphs.
- Future Work: The authors suggest incorporating realistic measurement noise models, scaling to larger graphs using low-rank or variational inference, handling unobserved confounders, and generalizing to broader classes of identifiable MNAR models.
Summary updated