Causally Consistent Normalizing Flow

Qingyang Zhou, Kangjie Lu, Meng XuAAAI 2025 · DOI

TL;DR

This paper introduces Causally Consistent Normalizing Flow (CCNF), a novel generative model that guarantees causal consistency without sacrificing expressiveness. CCNF achieves this by representing Structural Causal Models (SCMs) sequentially and employing partial causal transformations, enabling it to perform complex causal inference tasks and address real-world issues like unfairness.

What Problem Does This Paper Address?

Generative models (GMs) are increasingly used to model complex data distributions. When these GMs are informed by causal models (like SCMs), they can be used for causal inference tasks such as interventions and counterfactuals. However, a critical challenge arises when the causal graph implicitly learned by the GM is inconsistent with the explicitly defined causal graph of the SCM. This "causal inconsistency" can lead to significant societal issues, such as unfairness in decision-making systems. Prior attempts to enforce causal consistency in GMs, particularly in Normalizing Flows (NFs), have typically done so by severely limiting the model's architecture (e.g., restricting it to a single layer), which compromises its ability to approximate complex data distributions. This paper aims to bridge this gap by developing a causally consistent GM that retains its expressiveness.

Methods

The core innovation of CCNF lies in two novel constructs:

  • Sequential Representation of SCMs: Instead of directly modeling the entire SCM, CCNF first translates the SCM's causal graph into an ordered sequence. This is achieved by applying topological batching to the Directed Acyclic Graph (DAG) representing the causal structure. This sequential representation breaks down the complex causal dependencies into manageable steps.
  • Partial Causal Transformations: CCNF then utilizes normalizing flows with "partial causal transformations." These transformations allow the model to learn dependencies between variables in a structured way, respecting the causal order derived from the sequential representation. Crucially, these transformations are applied in a way that ensures causal consistency is maintained even when multiple layers are used in the normalizing flow architecture. This contrasts with previous methods that enforced consistency by eliminating intermediate layers.

By combining these, CCNF can approximate arbitrary complex distributions while inherently maintaining causal consistency.

Datasets

The paper evaluates CCNF on several synthetic datasets designed to represent various causal structures, including:

  • Nonlinear Triangle dataset
  • Nonlinear Simpson dataset
  • M-graph dataset
  • Network dataset
  • Backdoor dataset
  • Chain dataset

For real-world evaluation, the authors use the German credit dataset. This dataset is commonly used for predicting credit risk and is known to exhibit unfairness issues when classifiers directly use sensitive attributes like sex.

Key Insights and Ideas

  • Causal Consistency Without Sacrificing Expressiveness: The most significant insight is that causal consistency doesn't have to come at the cost of model expressiveness. CCNF demonstrates that by carefully structuring the generative process (sequential representation and partial transformations), one can achieve both.
  • Topological Batching as a Key Enabler: The paper highlights how topological batching, a standard graph algorithm, can be effectively repurposed to create a sequential representation of SCMs, which is fundamental to CCNF's design.
  • Unfairness Mitigation Through Counterfactuals: The real-world evaluation shows that CCNF can be used not only to model data but also to actively address societal issues like unfairness by generating counterfactual data or acting as a fairness-aware classifier.
  • The "Middle Layer" Problem Solved: The paper directly tackles the limitation of previous causally consistent models that couldn't use intermediate layers. CCNF shows that multiple layers can be incorporated while preserving causal integrity.

Key Results

  • Causal Consistency: CCNF consistently achieves a causal consistency score of 0 across various synthetic datasets, indicating perfect adherence to the underlying causal graph. Prior state-of-the-art models either failed to achieve this or could only do so with a single-layer architecture.
  • Causal Inference Performance: CCNF outperforms existing causally consistent models (CausalNF, VACA) in causal inference tasks (observations, interventions, counterfactuals) across multiple synthetic datasets, as measured by KL divergence, MMD, and RMSD.
  • Real-World Fairness: On the German credit dataset, CCNF significantly reduces individual unfairness from 9.00% to 0.00% while maintaining or even improving accuracy (from 73.00% to 75.80%).

Limitations and Future Work

  • Requirement of a Well-Defined Causal Graph: A primary limitation acknowledged is that CCNF, like many causal models, requires a pre-defined, accurate causal graph. Obtaining such graphs in real-world scenarios can be challenging.
  • Future Directions:
    • Handling Imperfect Causal Graphs: Extending CCNF to handle cases with incorrect or non-DAG causal graphs is a key future direction.
  • Causal Discovery: Leveraging CCNF's capabilities for causal discovery tasks (i.e., learning the causal graph itself) is another promising avenue.
  • Broader Applications: Exploring CCNF's utility in other domains beyond causal inference is also suggested.

Summary updated

BibTeX
@inproceedings{zhou2025causally,
  author = {Qingyang Zhou and Kangjie Lu and Meng Xu},
  title = {{Causally Consistent Normalizing Flow}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2025},
  doi = {10.1609/aaai.v39i21.34460},
}
Figure