TL;DR
This paper introduces Ante-Hoc Causal Explanations (AHCE), a novel framework for neural networks that explicitly models and quantifies both direct and indirect causal effects. By adding lateral feedforward connections between input neurons based on prior causal knowledge, the authors enable models to learn complex feature interactions that standard "black-box" architectures typically ignore.
What Problem Does This Paper Address?
Most explainability methods for neural networks (NNs) treat input features as independent entities. In this view, an NN is a function where each input contributes directly to the output. However, the real world is rarely that simple. Features often have causal relationships with one another—for example, a person's socioeconomic status might influence their level of education, which in turn affects their job role and eventual income.
Standard NNs, as shown in Figure 1(a), fail to capture these "indirect" pathways. Consequently, existing explanation methods (like Integrated Gradients or standard Causal Attributions) only quantify direct effects, potentially leading to biased or incomplete explanations. This is particularly problematic in safety-critical or fairness-sensitive domains where understanding the why behind a prediction is as important as the prediction itself.
Methods
The authors propose an ante-hoc approach, meaning causal structure is baked into the model architecture and training process rather than being analyzed after the fact.
- Augmented Architecture ($N^{Ind}$): They introduce "Layer 0" connections—lateral feedforward edges between input neurons that reflect known causal relationships (e.g., $X_1 \rightarrow X_2$). See Figure 2 for a comparison.
- Two-Phase Training: To ensure the model learns these relationships correctly, they use a two-phase algorithm. Phase 1 trains the main network while freezing the input-to-input connections. Phase 2 trains the entire model, using a regularization term to ensure the learned functions between inputs match the actual data distribution.
- Causal Quantification: They define and provide algorithms to calculate the Average Direct Causal Effect (ADCE) and Average Indirect Causal Effect (AICE).
- Scaling for High Dimensions: To handle complex models like Recurrent Neural Networks (RNNs), they employ a second-order Taylor approximation (using the Hessian) and a "binning" strategy that uses offline-computed statistics and clustering to speed up online explanation generation.
Datasets
The researchers evaluated AHCE across a diverse set of benchmarks:
- Synthetic Data: A custom dataset with known linear/nonlinear causal equations to establish ground truth.
- Real-World Benchmarks: Auto-MPG (predicting fuel efficiency), Lung Cancer (medical diagnosis), and Sachs (protein signaling pathways). Causal graphs for these are shown in Figure 4.
- Industry-Simulated Flight Data: Three time-series datasets (PBD, FD, MAD) from an industry-grade flight simulator used to test the efficiency of their RNN implementation.
Key Insights and Ideas
- Identifiability: The paper proves a critical hypothesis: indirect causal effects are only identifiable in an NN if there are explicit feedforward edges between input neurons in the architecture.
- Ante-hoc vs. Post-hoc: The authors argue that post-hoc methods are limited because they try to extract causal meaning from models that weren't built to respect causal structures. By using an ante-hoc approach, the model is forced to learn representations that align with real-world causal dynamics.
- Fairness and Bias: Modeling indirect effects is crucial for fairness. For instance, a sensitive attribute might have zero direct effect on a decision but a massive indirect effect through other features. AHCE allows researchers to pinpoint exactly where these biases propagate.
Key Results
- Accuracy: On synthetic data, AHCE achieved an average RMSE of 0.06 for total causal effects, significantly outperforming baselines like Causal Shapley (0.26) and Integrated Gradients (0.11).
- Real-World Performance: In the Sachs protein dataset, AHCE's explanations were much closer to the ground truth (RMSE 0.22) compared to other methods (ranging from 0.32 to 1.05).
- Efficiency: The proposed binning and Hessian-approximation strategies provided a 3x to 10x speedup in runtime for computing causal effects in complex RNN models without significant loss in precision.
Limitations and Future Work
- Reliance on Prior Knowledge: The current method requires a known (or partially known) causal graph. While the authors showed that even partial graphs help, the method's utility is tied to the quality of domain knowledge.
- No Latent Confounders: The framework assumes there are no unobserved variables influencing both the inputs and the output (Assumption 4.1).
- Future Directions: The authors suggest extending this work to causal discovery, where the model could potentially learn the structure of the causal graph from the data itself, rather than relying on it as a prior.
Summary updated