Subgraph Aggregation for Out-of-Distribution Generalization on Graphs

B Liu, Haoyang Li, Shuning Wang, Suping Nie, Shanghang Zhang

Abstract

Out-of-distribution (OOD) generalization in Graph Neural Networks (GNNs) has gained significant attention due to its critical importance in graph-based predictions in real-world scenarios. Existing methods primarily focus on extracting a single causal subgraph from the input graph to achieve generalizable predictions. However, relying on a single subgraph can lead to susceptibility to spurious correlations and is insufficient for learning invariant patterns behind graph data. Moreover, in many real-world applications, such as molecular property prediction, multiple critical subgraphs may influence the target label property. To address these challenges, we propose a novel framework, SubGraph Aggregation (SuGAr), designed to learn a diverse set of subgraphs that are crucial for OOD generalization on graphs. Specifically, SuGAr employs a tailored subgraph sampler and diversity regularizer to extract a diverse set of invariant subgraphs. These invariant subgraphs are then aggregated by averaging their representations, which enriches the subgraph signals and enhances coverage of the underlying causal structures, thereby improving OOD generalization. Extensive experiments on both synthetic and real-world datasets demonstrate that SuGAr outperforms state-of-the-art methods, achieving up to a 24% improvement in OOD generalization on graphs. To the best of our knowledge, this is the first work to study graph OOD generalization by learning multiple invariant subgraphs.

Introduction

Graph Neural Networks (GNNs) often struggle when the data they encounter at test time differs from their training distribution—a challenge known as Out-of-Distribution (OOD) generalization. A common strategy to address this is "invariant learning," which attempts to identify the "causal" subgraph (the part of the graph truly responsible for the label) while ignoring "spurious" correlations (noise that happens to correlate with the label in the training set).

However, current approaches typically assume there is only one invariant subgraph. In reality, many graphs—especially molecules—contain multiple critical structures. For example, the activity of an Aspirin molecule is determined by two distinct functional groups: a carboxyl group and an ester group. If a model only captures one, its predictions will be suboptimal and fragile. SuGAr addresses this by learning to identify and aggregate multiple diverse invariant subgraphs.

Motivation: Capturing multiple functional groups (e.g., in Aspirin) is essential for robust OOD prediction.

The SuGAr Framework

SuGAr is a framework designed to identify and combine multiple diverse invariant subgraphs through three main components: diversity injection, invariant learning, and subgraph aggregation.

Overview of the SuGAr architecture, illustrating the pipeline from sampling to aggregation.

1. Diversity Injection

To prevent multiple models from converging on the same subgraph, SuGAr uses a two-pronged approach. First, it employs a subgraph sampler that randomly drops edges during training. Second, it introduces a Diversity Regularizer ($\mathcal{L}_{div}$), which explicitly penalizes models if their predicted edge weights (the "importance" they assign to different parts of the graph) are too similar.

2. Invariant Learning

Each base model is trained using a contrastive loss to ensure the subgraphs it identifies are truly invariant across different environments, following the principles of Causal Invariant Graph Learning (CIGA).

3. Subgraph Aggregation

After training $n$ diverse models, SuGAr offers two ways to combine their knowledge:

  • SuGAr (ENS): An ensemble method that averages the edge weights from all models to form a comprehensive "multiple invariant subgraph" ($G_c^m$) and then uses soft voting for the final classification.
  • SuGAr (WA): A weight-averaging approach that merges the parameters of the base models into a single model, improving performance without increasing inference-time computation.

Experimental Evaluation

The researchers evaluated SuGAr across 15 different datasets, including synthetic benchmarks like SPMotif and the new SUMotif (designed for multiple-motif scenarios), and real-world benchmarks like DrugOOD, CMNIST-sp, and Twitter sentiment analysis.

Key insights from the experiments include:

  • Diversity is Non-Trivial: Without the diversity regularizer, independent models often "lazy-load" onto the same easy-to-find patterns. Forcing them to look elsewhere allows the model to find missing causal pieces.
  • Aggregation as Completion: By aggregating multiple subgraphs, the model effectively "fills in the blanks." If Model A finds functional group 1 and Model B finds functional group 2, the aggregated representation is far more robust than either alone.

Results and Impact

SuGAr outperformed all baselines (including ERM, IRM, and GSAT) across almost all 15 datasets. On the synthetic SUMotif dataset, SuGAr improved OOD accuracy by 30% compared to previous state-of-the-art invariant learning methods. On the DrugOOD benchmark, which mimics real-world drug discovery challenges, SuGAr consistently achieved the highest ROC-AUC, demonstrating its practical utility in bioinformatics. Overall, the framework achieves up to a 24% improvement in OOD generalization on graphs.

Conclusion

SuGAr represents a significant step forward in graph OOD generalization by moving beyond the single-subgraph assumption. By explicitly encouraging and aggregating diverse invariant subgraphs, it provides a more robust and comprehensive understanding of complex graph structures. Future work may explore dynamic methods for determining the optimal number of subgraphs for different graph types and further optimizing the computational overhead of training multiple models.

Summary updated

Abstract

Out-of-distribution (OOD) generalization in Graph Neural Networks (GNNs) has gained significant attention due to its critical importance in graph-based predictions in real-world scenarios. Existing methods primarily focus on extracting a single causal subgraph from the input graph to achieve generalizable predictions. However, relying on a single subgraph can lead to susceptibility to spurious correlations and is insufficient for learning invariant patterns behind graph data. Moreover, in many real-world applications, such as molecular property prediction, multiple critical subgraphs may influence the target label property. To address these challenges, we propose a novel framework, SubGraph Aggregation (SuGAr), designed to learn a diverse set of subgraphs that are crucial for OOD generalization on graphs. Specifically, SuGAr employs a tailored subgraph sampler and diversity regularizer to extract a diverse set of invariant subgraphs. These invariant subgraphs are then aggregated by averaging their representations, which enriches the subgraph signals and enhances coverage of the underlying causal structures, thereby improving OOD generalization. Extensive experiments on both synthetic and real-world datasets demonstrate that \ours outperforms state-of-the-art methods, achieving up to a 24% improvement in OOD generalization on graphs. To the best of our knowledge, this is the first work to study graph OOD generalization by learning multiple invariant subgraphs. code: https://github.com/Nanolbw/SuGAr
BibTeX
@inproceedings{liu2025subgraph,
  author = {B Liu and Haoyang Li and Shuning Wang and Suping Nie and Shanghang Zhang},
  title = {{Subgraph Aggregation for Out-of-Distribution Generalization on Graphs}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2025},
  doi = {10.1609/aaai.v39i18.34065},
}
Figure