Introduction
The integration of Large Language Models (LLMs) with Graph Neural Networks (GNNs) has become a dominant paradigm in graph representation learning. In this "LLM-enhancer-plus-GNN" setup, the LLM typically serves as a sophisticated feature extractor for text-rich nodes, while the GNN models the structural dependencies and topology of the graph. Despite its popularity, this architecture remains a "black box." It is unclear how information is partitioned between the two components or how the GNN utilizes the high-dimensional features provided by the LLM. This paper addresses this gap by applying causal inference techniques to identify the underlying mechanisms of these hybrid models.
Causal Abstraction Framework
To interpret the internal logic of the LLM-GNN pipeline, the authors utilize causal abstraction. This involves treating the complex neural network as a "low-level" model and attempting to align its internal states with a "high-level" causal model representing known logical rules.
By establishing a "match" between specific hidden layers in the neural network and variables in a causal graph, researchers can pinpoint which parts of the architecture are responsible for specific reasoning steps.
Interchange Intervention Methodology
The core technique used to validate these mappings is interchange intervention. This process involves swapping internal hidden variables between two different graph inputs and observing if the resulting change in the network's output mirrors the change predicted by the high-level causal model.
The methodology follows a four-step process:
- Data Generation: Creating synthetic graphs (using the novel CCSG dataset) with known causal rules.
- High-level Output: Calculating the expected output $h(\cdot)$ based on ground-truth logic.
- Low-level Output: Running the neural network $f(\cdot)$ and performing interventions on hidden layers.
- Loss Calculation: Minimizing the Interchange Intervention Loss ($L_{II}$) to align the neural network's behavior with the causal model.
Key Insights from Causal Analysis
The application of this framework yielded several critical insights:
- LLMs as Feature Extractors: In standard configurations where LLM parameters are fixed, their role is primarily restricted to node-level feature extraction. They do not contribute significantly to structural or multi-hop causal reasoning.
- GNN Structural Stability: Increasing the size or depth of a GNN does not fundamentally alter its internal causal logic. Scaling primarily increases capacity without changing the underlying way the model processes graph structure.
- The Transmission Bottleneck: The primary performance bottleneck lies in the "bridge"—how features are passed from the LLM to the GNN. The choice of which token features to transmit significantly impacts the GNN's ability to perform downstream tasks.
The Attention-based Transmission (AT) Module
Based on these findings, the authors developed the Attention-based Transmission (AT) module. Rather than passing a single, static feature vector from the LLM, the AT module employs a transformer encoder and an attention mechanism. This allows the system to dynamically select and weight the most relevant features from multiple LLM output tokens across various prompts, optimizing the information flow to the GNN.
Experimental Results
The AT module was evaluated on standard benchmarks (Cora, Pubmed, Instagram) using various LLM backbones (Llama2, Qwen2, Llama3) and GNN architectures (GCN, GAT, GraphSAGE).
- Performance Gains: The AT module consistently improved accuracy. For instance, on the Pubmed dataset, Llama3 + GraphSAGE saw a +3.09% improvement, while Llama2 + GCN improved by +2.64%.
- Versatility: The module proved to be a robust, plug-and-play enhancer across different model combinations.
Limitations and Future Work
While providing deep insights, the study focuses on fixed-parameter LLMs. Future research could explore end-to-end fine-tuning, which might reveal different causal dynamics. Additionally, the current analysis operates at node-level granularity; investigating sub-node text interactions remains an open area. The authors suggest that this causal abstraction framework is broadly applicable to other complex, multi-modal architectures beyond graphs.
Summary updated