Abstract
Deep learning models are often criticized for being sophisticated correlation engines that fail to account for causal relationships. This paper introduces Causal Graph Routing (CGR), a novel framework that integrates active causal intervention into neural architectures. By dynamically routing data through specialized deconfounding blocks, CGR mitigates dataset biases and outperforms larger, correlation-based models in Visual Question Answering (VQA) and Long Document Classification (LDC) with significantly fewer parameters.
Introduction: Beyond Correlation
While "correlation is not causation" is a fundamental principle, most modern AI models, including Transformers, primarily function as pattern matchers. This reliance on correlation leads to confounding, where a model learns spurious associations—such as linking "rooster crowing" with "sunrise"—without understanding the underlying mechanism. In practical applications like VQA, this manifests as a tendency to answer based on dataset frequency (e.g., predicting "sunny" for any weather query) rather than visual evidence.
Methodology: Causal Graph Routing (CGR)
The CGR framework moves beyond static architectures by treating causal inference as a dynamic routing problem. The architecture consists of stacked layers, each containing three parallel deconfounding blocks designed to address specific causal scenarios:
- No Confounder Block: A direct path where input predicts output via a mediator.
- Back-door Adjustment Block: Designed to block the influence of observable confounders, such as external common-sense knowledge.
- Front-door Adjustment Block: Mitigates unobservable confounders by focusing on causal flow through known mediators, such as specific visual attributes.
Dynamic Selection via Sufficient Cause
The core innovation of CGR is the use of the Sufficient Cause concept to weight these blocks. Instead of applying a single causal adjustment to an entire task, CGR dynamically routes each sample through the most appropriate graph. This is achieved through a sharpening softmax function during training, which allows the model to explore various paths before converging on the most effective causal mechanism for a given input.
Experimental Evaluation
The researchers evaluated CGR across three diverse benchmarks: VQA2.0 (visual question-answering), ECtHR (legal document classification), and 20 NewsGroups (topic classification).
Key Results
CGR demonstrated superior performance and efficiency:
- VQA Accuracy: CGR achieved 75.47% on VQA2.0, surpassing the previous state-of-the-art transformer model, TRAR (72.93%).
- NLP Performance: On the ECtHR dataset, CGR reached a 76.6 Macro F1 score, outperforming specialized models like Legal-BERT.
- Parameter Efficiency: Remarkably, CGR outperformed ERNIE-VIL (large) using only 6 causal layers compared to ERNIE-VIL's 30-layer stack, suggesting that causal intelligence is more efficient than brute-force correlation.
Discussion and Key Insights
Qualitative analysis revealed that CGR adapts its causal strategy across layers. In VQA tasks, early layers often utilize front-door adjustments to handle visual biases, while later layers shift to back-door adjustments to integrate external knowledge. This hierarchical causal processing allows the model to build a more robust semantic understanding.
Limitations and Future Work
Currently, CGR requires pre-defined causal graphs, which necessitates domain expertise to identify mediators and confounders. Future research aims to develop pre-trained causal models that can autonomously map cause-and-effect relationships, potentially leading to AI that generalizes better to out-of-distribution scenarios.
Summary updated