GraphPPD: Posterior Predictive Modelling for Graph-Level Inference

Soumyasundar Pal, Liheng Ma, Amine Natik, Yingxue Zhang, Mark CoatesarXiv 2025 · DOI · arXiv

Introduction

Graph Neural Networks (GNNs) have established themselves as the standard for analyzing network-structured data. However, like many deep learning architectures, they are prone to overconfidence, often failing to provide reliable uncertainty estimates. While significant research has addressed uncertainty at the node and link levels, graph-level inference—critical for fields like drug discovery and protein analysis—remains underserved. Existing Bayesian approaches for graph-level tasks are frequently computationally prohibitive or struggle to model the intricate relationship between complex graph structures and their associated labels.

The GraphPPD Framework

To address these challenges, we introduce GraphPPD (Posterior Predictive Distribution), a variational framework designed to quantify predictive uncertainty in graph-level tasks. The core innovation lies in learning an amortized approximation of the PPD, which represents the probability of a label given a new input and the context of the training set.

Architecture and Workflow

The GraphPPD architecture is composed of two primary components: a deterministic feature extractor and an amortized PPD module. The feature extractor (e.g., GINE or GMT) maps a graph to a fixed-length embedding. The PPD module then utilizes a cross-attention mechanism to compare the embedding of a "target" graph against a "context" set of labeled examples from the training data.

Overview of the GraphPPD training and inference pipelines.

During inference, the model adaptively forms predictions by attending to relevant labeled examples. This "context-aware" approach allows the model to ground its predictions in previously seen data, providing a natural mechanism for estimating confidence. GraphPPD supports both end-to-end training and a more efficient two-stage approach where a pre-trained encoder is frozen, significantly reducing the computational overhead.

Experimental Evaluation

GraphPPD was evaluated across 18 benchmarks, including 7 Open Graph Benchmark (OGB) datasets, 10 TU datasets, and the ZINC-12k regression task. These datasets cover a wide range of domains, from molecular property prediction to social network analysis.

Key Performance Insights

  • Accuracy Improvements: GraphPPD consistently outperforms baseline GNNs. For instance, when paired with a Graph Multiset Transformer (GMT) on OGB datasets, it achieved a 3.93% relative improvement in ROC-AUC.
  • Reliable Uncertainty: Unlike Monte Carlo (MC) Dropout, which can sometimes degrade performance in graph classification, GraphPPD provides robust uncertainty estimates that translate into better predictive accuracy.
  • Selective Prediction: A key strength of GraphPPD is its performance in "selective prediction" scenarios. By using the model's uncertainty as a signal, difficult cases can be identified and deferred to a human expert (an "oracle").
Collaborative ROC-AUC performance showing how GraphPPD (Ours) effectively utilizes human intervention compared to the GMT baseline.

As shown in the collaborative performance plots, GraphPPD's uncertainty metric effectively identifies graphs likely to be misclassified, leading to higher overall accuracy as the fraction of reviewed queries increases.

Conclusion and Future Directions

GraphPPD offers a scalable and effective solution for graph-level uncertainty quantification. By treating the training set as a dynamic context during inference, it achieves state-of-the-art results with greater efficiency than traditional deep ensembles. While the addition of the PPD module introduces some overhead, the performance gains and reliability make it a compelling choice for high-stakes applications like molecular discovery. Future work will explore extending this context-based predictive modeling to other domains such as computer vision and natural language processing.

Summary updated

Abstract

Accurate modelling and quantification of predictive uncertainty is crucial in deep learning since it allows a model to make safer decisions when the data is ambiguous and facilitates the users' understanding of the model's confidence in its predictions. Along with the tremendously increasing research focus on \emph{graph neural networks} (GNNs) in recent years, there have been numerous techniques which strive to capture the uncertainty in their predictions. However, most of these approaches are specifically designed for node or link-level tasks and cannot be directly applied to graph-level learning problems. In this paper, we propose a novel variational modelling framework for the \emph{posterior predictive distribution}~(PPD) to obtain uncertainty-aware prediction in graph-level learning tasks. Based on a graph-level embedding derived from one of the existing GNNs, our framework can learn the PPD in a data-adaptive fashion. Experimental results on several benchmark datasets exhibit the effectiveness of our approach.
BibTeX
@misc{pal2025graphppd,
  author = {Soumyasundar Pal and Liheng Ma and Amine Natik and Yingxue Zhang and Mark Coates},
  title = {{GraphPPD: Posterior Predictive Modelling for Graph-Level Inference}},
  year = {2025},
  eprint = {2508.16995},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2508.16995},
}
Figure