In-Distribution Explanations for Graph Neural Networks

This direction is split into other directions: out-of-distribution explanation bias. Its last version stays readable and citable here.

Version 1 of 126 papers49 conceptsfrozen 2026-10-04

Graph Neural Networks (GNNs) are often treated as black boxes, but understanding their decisions is critical for high-stakes applications like drug discovery or financial fraud detection. Most explanation methods attempt to identify a "rationalizing" subgraph—a small subset of nodes and edges that supposedly drove the model's prediction.

However, a recurring problem across recent research is that these isolated subgraphs often look nothing like the data the model saw during training. This creates a distribution shift that renders the resulting explanations unreliable.

The Problem of Broken Topologies

When we explain a GNN by isolating a subgraph, we are essentially performing a "surgical" operation on the input. As Lukas Faber and Roger Wattenhofer point out in Contrastive Graph Neural Network Explanation, removing edges or nodes often creates unrealistic, out-of-distribution (OOD) graphs.

To a GNN trained on complete, complex structures, a tiny isolated fragment can act more like an adversarial attack than a meaningful input. Because the model was never trained to process these "broken" topologies, its output on the fragment might be nonsensical. If the model produces the "correct" label on a fragment, it might be doing so for the wrong reasons—perhaps triggered by a structural artifact rather than the actual features. Dongsheng Luo and colleagues argue in Towards Faithful and Consistent Explanations for Graph Neural Networks that this shift leads to "spurious" explanations that do not reflect the model's true decision-making process.

Restoring Context via Proxy Graphs

To fix this, researchers have moved toward "distribution-compliant" explanations. The core intuition is simple: if the model cannot be trusted on fragments, we must evaluate those fragments within a realistic context.

One prominent approach is the use of Proxy Graphs. In Generating In-Distribution Proxy Graphs for Explaining Graph Neural Networks, Zhuomin Chen and Jiaxing Zhang propose embedding a candidate explanatory subgraph into a generated, realistic "background" structure. By placing the explanation back into a familiar data distribution, the GNN can evaluate it reliably. This "in-distribution" evaluation has been shown to improve explanation accuracy (AUC-ROC) by over 10% on molecular datasets.

Similarly, Jialin Chen and Rex Ying introduced D4Explainer, which uses discrete denoising diffusion to learn the underlying data manifold. Instead of just pruning edges, it generates counterfactual and model-level explanations that are statistically realistic, ensuring the GNN's response is a reaction to meaningful structural features rather than noise.

Rethinking Fidelity and Evaluation

The OOD problem doesn't just affect how we generate explanations; it also breaks how we measure them. Standard "fidelity" metrics—which measure how much a model's prediction drops when an explanation is removed—are often confounded by structural novelty. A drop in performance might occur because the model is confused by a fragmented input, not because the removed features were actually important.

To address this, Xu Zheng and Farhad Shirani developed Robust Fidelity measures. Instead of deterministic removal, they use stochastic edge sampling to keep the modified graphs within the support of the original training distribution.

In a similar vein, Kenza Amara and Rex Ying proposed GInX-Eval, which fine-tunes models on modified graphs to ensure they remain "in-distribution" during evaluation. Their findings were sobering: many popular explainers performed no better than random when the OOD bias was removed, highlighting the urgent need for distribution-aware methods.

Beyond Classification: The Regression Challenge

The distribution shift is even more pronounced in regression tasks, where the model must predict continuous values (like molecular solubility) rather than discrete labels. Jiaxing Zhang addressed this in RegExplainer by using a "graph mix-up" strategy. By blending explanatory subgraphs with other real graph structures, the method creates a continuous path of in-distribution samples, allowing the explainer to accurately estimate the "causal" contribution of specific motifs to the final regression value.

Summary of Key Mechanisms

Mechanism Research Perspective Core Idea
Generative Context D4Explainer Use diffusion models to ensure explanations stay on the data manifold.
Proxy Backgrounds ProxyExplainer Embed subgraphs into realistic "supergraphs" to avoid OOD errors.
Graph Mix-up RegExplainer Use data augmentation to bridge the gap between fragments and full graphs.
Stochastic Sampling Robust Fidelity Evaluate importance by sampling edges rather than hard pruning.

By prioritizing distribution compliance, this research direction ensures that GNN explanations are faithful: they reflect what the model actually learned, rather than how the model fails when faced with broken data.

Papers in this version

History & citationVersion 1 · 2026-10-04

Split into other directions · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_in_distribution_explanations_for_graph_n_be9ee412,
  title = {In-Distribution Explanations for Graph Neural Networks},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/in-distribution-explanations-for-graph-neural-networks/rdr_be9ee4128df84278ba8c03f150c89302},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-19},
}

This exact version (version 1, frozen text):

@misc{lacuna_in_distribution_explanations_for_graph_n_be9ee412_v1,
  title = {In-Distribution Explanations for Graph Neural Networks},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/in-distribution-explanations-for-graph-neural-networks/rdr_be9ee4128df84278ba8c03f150c89302/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-19},
}