Bayesian Causal Structural Learning with Zero-Inflated Poisson Bayesian Networks

Junsouk Choi, Robert S. Chapkin, Yang NiNeurIPS 2020

Introduction

Researchers have developed a new Bayesian network model, the Zero-Inflated Poisson Bayesian Network (ZIPBN), specifically designed to uncover causal relationships in "zero-inflated" count data. These are datasets where zeros appear far more frequently than standard statistical models expect. By proving that this model is mathematically "identifiable," the authors demonstrate that we can determine the direction of causality from a single snapshot of observational data—a significant breakthrough for fields like single-cell genomics and economics.

The Challenge: Zero-Inflation and Markov Equivalence

In many real-world datasets, such as the number of mRNA molecules in a single cell or insurance claim counts, we observe a massive number of zeros. Standard statistical tools like the Poisson distribution fail here because they cannot account for this extreme "zero-inflation."

Furthermore, when trying to learn causal structures (who influences whom) from observational data using Bayesian Networks (BNs), researchers hit a wall known as "Markov Equivalence." Essentially, different causal directions (e.g., $A \rightarrow B$ vs. $B \rightarrow A$) often look identical to an algorithm analyzing a static dataset.

Examples of Markov equivalence classes in Bayesian Networks.

To tell these equivalent structures apart, researchers usually have to make very strong assumptions or conduct expensive, time-consuming experiments. This paper asks a pivotal question: Can we use the unique statistical "shape" of zero-inflated count data to automatically figure out the direction of causality?

The ZIPBN Approach

To solve this, the authors introduce the ZIPBN model, which leverages the zeros rather than treating them as a nuisance.

  1. The Model: Each variable in the network is modeled using a zero-inflated Poisson regression. It uses two components: one that generates "extra" zeros and a standard Poisson component for the rest of the counts. Crucially, both components are influenced by the variable's "parents" in the causal network.
  2. Identifiability Proof: The core theoretical contribution is a proof showing that ZIPBNs are identifiable. Unlike many other models, the "correct" causal graph produces a unique probability distribution that cannot be perfectly mimicked by a graph with reversed edges.
  3. Fully Bayesian Inference: Instead of just looking for a single "best" graph, the authors use a Bayesian approach to estimate the probability of every possible edge, providing a measure of confidence for each causal link.
  4. Parallel Tempering MCMC: Learning a network structure is a complex optimization problem where algorithms can easily get stuck in "local" solutions. To overcome this, they use parallel tempering, running multiple versions of the search at different "temperatures." This allows the algorithm to jump out of local traps and explore the entire landscape of possible networks.

Key Theoretical Insights

  • The "Zero" is a Signal: Usually, zeros are seen as missing data or noise. Here, the specific pattern of zeros provides the mathematical "hook" needed to distinguish the direction of causality.
  • No "Faithfulness" Required: Many causal discovery algorithms rely on a controversial assumption called "faithfulness" (that the data perfectly reflects the underlying graph's independencies). The authors show that ZIPBN doesn't need this assumption because the discrete count nature of the data prevents the "accidental cancellations" of effects that plague continuous models.
  • Automatic Complexity Control: By using a specific "beta-like" prior for edge inclusion, the model automatically penalizes overly complex networks, preventing it from "hallucinating" causal links as the number of variables grows.

Experimental Validation and Results

The model was rigorously tested on both synthetic datasets (varying sample sizes, variables, and zero-inflation levels) and real-world single-cell RNA sequencing (scRNA-seq) data from mouse intestinal cells.

Superior Accuracy in Simulations

In synthetic simulations, ZIPBN consistently outperformed existing methods like OverDispersion Scoring (ODS) and Moments Ratio Scoring (MRS), especially as the number of variables increased. It maintained a much lower False Discovery Rate (FDR) while achieving higher True Positive Rates (TPR).

Simulation results comparing ZIPBN against ODS and MRS across varying sample sizes and variable counts.

Real-World Biological Discovery

When tested on a validation set of 479 known "transcription factor and target" gene pairs from the TRRUST database, ZIPBN correctly identified 304 causal relationships. This was significantly better than random guessing ($p < 1.1 \times 10^{-9}$) and vastly outperformed the competing MRS method, which found only 198.

Furthermore, in an analysis of 40 genes within the well-studied Wnt signaling pathway, the model identified 60 edges. Many of these (like the regulation of Jun by Ctnnb1) are well-documented in biological literature, and the model successfully pinpointed known "hub genes" central to cell signaling.

Limitations and Future Work

While powerful, the ZIPBN approach has some limitations:

  • Computational Intensity: MCMC-based methods are notoriously slow compared to "greedy" search algorithms. While parallel tempering makes the search more robust, it requires significant CPU time (e.g., analyzing the 40-gene pathway took 1.7 hours).
  • The "Peaky" Mode Problem: As the sample size ($n$) gets very large, the "modes" (the best solutions) become very sharp, making it harder for the MCMC algorithm to transition between them. Massive datasets may require even more parallel chains.
  • Model Misspecification: The model performs best when the data is truly generated by a zero-inflated Poisson process. If the data follows a different distribution (like a Zero-Inflated Negative Binomial), the performance gap between ZIPBN and other methods narrows.

Summary updated

BibTeX
@inproceedings{choi2020bayesian,
  author = {Junsouk Choi and Robert S. Chapkin and Yang Ni},
  title = {{Bayesian Causal Structural Learning with Zero-Inflated Poisson Bayesian Networks}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {2020},
}
Figure