Towards Learning and Explaining Indirect Causal Effects in Neural Networks

Abbavaram Gowtham Reddy, Saketh Bachu, Harsharaj Pathak, Benin L Godfrey, Vineeth N. Balasubramanian, Varshaneya V, Satya Narayanan Kar

TL;DR

This paper introduces Ante-Hoc Causal Explanations (AHCE), a novel framework for neural networks that explicitly models and quantifies both direct and indirect causal effects. By adding lateral feedforward connections between input neurons based on prior causal knowledge, the authors enable models to learn complex feature interactions that standard "black-box" architectures typically ignore.

What Problem Does This Paper Address?

Most explainability methods for neural networks (NNs) treat input features as independent entities. In this view, an NN is a function where each input contributes directly to the output. However, the real world is rarely that simple. Features often have causal relationships with one another—for example, a person's socioeconomic status might influence their level of education, which in turn affects their job role and eventual income.

Standard NNs, as shown in Figure 1(a), fail to capture these "indirect" pathways. Consequently, existing explanation methods (like Integrated Gradients or standard Causal Attributions) only quantify direct effects, potentially leading to biased or incomplete explanations. This is particularly problematic in safety-critical or fairness-sensitive domains where understanding the why behind a prediction is as important as the prediction itself.

Figure 1: (a) A marginalized NN whose inputs S, E, R are not causally related. (b) A marginalized NN whose inputs are con- nected through feedforward connections (e.g., S →E) to capture underlying causal relationships (e.g., S causes E) to learn the indi- rect causal effects of inputs on output (e.g. effect of S on I via E).

Methods

The authors propose an ante-hoc approach, meaning causal structure is baked into the model architecture and training process rather than being analyzed after the fact.

  1. Augmented Architecture ($N^{Ind}$): They introduce "Layer 0" connections—lateral feedforward edges between input neurons that reflect known causal relationships (e.g., $X_1 \rightarrow X_2$). See Figure 2 for a comparison.
  2. Two-Phase Training: To ensure the model learns these relationships correctly, they use a two-phase algorithm. Phase 1 trains the main network while freezing the input-to-input connections. Phase 2 trains the entire model, using a regularization term to ensure the learned functions between inputs match the actual data distribution.
  3. Causal Quantification: They define and provide algorithms to calculate the Average Direct Causal Effect (ADCE) and Average Indirect Causal Effect (AICE).
  4. Scaling for High Dimensions: To handle complex models like Recurrent Neural Networks (RNNs), they employ a second-order Taylor approximation (using the Hessian) and a "binning" strategy that uses offline-computed statistics and clustering to speed up online explanation generation.
Figure 2: Comparison of the proposed architecture N Ind with a traditional NN architecture N. G is the ground truth causal graph. N and N Ind differ in input layer such that the inputs in N Ind are connected (shown in blue color) according to the causal edges in G. In contrast, the inputs in N are independent. N and N Ind may contain edges that are not present in G due to the feedforward con- nections from input layer to predictions in NN architecture (e.g., X1 →ˆY is present in N, N Ind but not in G).

Datasets

The researchers evaluated AHCE across a diverse set of benchmarks:

  • Synthetic Data: A custom dataset with known linear/nonlinear causal equations to establish ground truth.
  • Real-World Benchmarks: Auto-MPG (predicting fuel efficiency), Lung Cancer (medical diagnosis), and Sachs (protein signaling pathways). Causal graphs for these are shown in Figure 4.
  • Industry-Simulated Flight Data: Three time-series datasets (PBD, FD, MAD) from an industry-grade flight simulator used to test the efficiency of their RNN implementation.
Figure 4: Causal graphs for Auto-MPG, Lung Cancer, and Sachs datasets.

Key Insights and Ideas

  • Identifiability: The paper proves a critical hypothesis: indirect causal effects are only identifiable in an NN if there are explicit feedforward edges between input neurons in the architecture.
  • Ante-hoc vs. Post-hoc: The authors argue that post-hoc methods are limited because they try to extract causal meaning from models that weren't built to respect causal structures. By using an ante-hoc approach, the model is forced to learn representations that align with real-world causal dynamics.
  • Fairness and Bias: Modeling indirect effects is crucial for fairness. For instance, a sensitive attribute might have zero direct effect on a decision but a massive indirect effect through other features. AHCE allows researchers to pinpoint exactly where these biases propagate.

Key Results

  • Accuracy: On synthetic data, AHCE achieved an average RMSE of 0.06 for total causal effects, significantly outperforming baselines like Causal Shapley (0.26) and Integrated Gradients (0.11).
  • Real-World Performance: In the Sachs protein dataset, AHCE's explanations were much closer to the ground truth (RMSE 0.22) compared to other methods (ranging from 0.32 to 1.05).
  • Efficiency: The proposed binning and Hessian-approximation strategies provided a 3x to 10x speedup in runtime for computing causal effects in complex RNN models without significant loss in precision.

Limitations and Future Work

  • Reliance on Prior Knowledge: The current method requires a known (or partially known) causal graph. While the authors showed that even partial graphs help, the method's utility is tied to the quality of domain knowledge.
  • No Latent Confounders: The framework assumes there are no unobserved variables influencing both the inputs and the output (Assumption 4.1).
  • Future Directions: The authors suggest extending this work to causal discovery, where the model could potentially learn the structure of the causal graph from the data itself, rather than relying on it as a prior.

Summary updated

Abstract

Recently, there has been a growing interest in learning and explaining causal effects within Neural Network (NN) models. By virtue of NN architectures, previous approaches consider only direct and total causal effects assuming independence among input variables. We view an NN as a structural causal model (SCM) and extend our focus to include indirect causal effects by introducing feedforward connections among input neurons. We propose an ante-hoc method that captures and maintains direct, indirect, and total causal effects during NN model training. We also propose an algorithm for quantifying learned causal effects in an NN model and efficient approximation strategies for quantifying causal effects in high-dimensional data. Extensive experiments conducted on synthetic and real-world datasets demonstrate that the causal effects learned by our ante-hoc method better approximate the ground truth effects compared to existing methods.
BibTeX
@inproceedings{reddy2024towards,
  author = {Abbavaram Gowtham Reddy and Saketh Bachu and Harsharaj Pathak and Benin L Godfrey and Vineeth N. Balasubramanian and Varshaneya V and Satya Narayanan Kar},
  title = {{Towards Learning and Explaining Indirect Causal Effects in Neural Networks}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2024},
  doi = {10.1609/aaai.v38i13.29399},
}
Figure