Empowering Federated Graph Rationale Learning with Latent Environments

Linan Yue, Qi Liu, Yawen Li, Fangzhou Yao, Weibo Gao, Junping Du

TL;DR

This paper introduces EaDA, a novel federated learning framework for graph rationalization that tackles data heterogeneity and local shortcut learning. By inferring and leveraging "latent environments" across clients, EaDA generates more robust rationales, significantly improving prediction performance in privacy-preserving settings.

What Problem Does This Paper Address?

Graph Neural Networks (GNNs) are powerful for graph classification, but understanding why they make a prediction is crucial. Graph rationalization aims to identify the specific subgraph (the "rationale") that supports a GNN's decision. However, many existing rationalization methods rely on centralized datasets, which is problematic for sensitive data like molecular properties where privacy is paramount. Federated Learning (FL) offers a solution by allowing collaborative training without sharing raw data.

The challenge in Federated Graph Rationalization is data heterogeneity. Different clients in an FL system often have non-independent and identically distributed (non-IID) data, meaning their data distributions vary. This heterogeneity exacerbates a common issue in rationalization: models exploiting "shortcuts" – spurious correlations in the data that help make predictions but aren't causally linked to the true outcome. These shortcuts are often environment-specific, meaning they differ across clients, leading to poor global model performance. Previous attempts to address this, like FedGR, are computationally expensive.

Methods

The proposed solution, Environment-aware Data Augmentation (EaDA), tackles these issues with two core components:

  • Environment-aware Rationale Extraction (ERE):
    • Latent Environment Inference: Instead of assuming environments are unavailable (like FedGR), EaDA infers them. It uses prototype learning on the complement subgraphs (nodes not part of the rationale) to capture environment-specific features. These inferred "environments" are then aggregated on the server to form a global environment.
  • Counterfactual Data Generation: Clients use this global environment information to generate counterfactual samples. This involves transforming local rationale representations into different environments, creating new data points that disrupt spurious correlations and make the model more robust. The labels of these generated samples remain unchanged.
  • Local-Global Alignment (LGA):
    • Contrastive Learning: This module uses contrastive learning to align the local rationale representations with the global rationale representations. This helps the local models learn from the global information, further mitigating the performance degradation caused by data heterogeneity.

The overall training process involves clients performing ERE and LGA, then sending updated models and inferred environments to the server for aggregation.

Datasets

The paper evaluates EaDA on:

  • Synthetic Dataset: Spurious-Motif, a benchmark designed to introduce shortcuts by varying the relationship between motif subgraphs (rationales) and base subgraphs (complements). Datasets with different levels of bias (b=0.5, 0.7, 0.9) were used.
  • Real-world Datasets: Several datasets from the Open Graph Benchmark (OGB), specifically MolHIV, MolToxCast, MolBBBP, MolBACE, and MolSIDER. These datasets are commonly used for molecular property prediction.

For FL scenarios, the authors employ a Latent Dirichlet Allocation (LDA) based partitioning to create heterogeneous data distributions across clients.

Key Insights and Ideas

  • Inferring Environments is Key: The core insight is that instead of assuming environments are unknown or trying to perfectly reconstruct them, inferring abstract "latent environments" is sufficient and more practical for federated graph rationalization.
  • Complement Subgraphs as Environment Indicators: The paper cleverly uses the complement subgraph (non-rationale parts) to infer the environment. This is because these parts are more likely to capture variations and discriminative features related to the data distribution.
  • Counterfactuals for Robustness: Generating counterfactual data by transforming local rationales into different inferred environments is a powerful way to break spurious correlations and improve the faithfulness of extracted rationales.
  • LGA Bridges the Gap: The Local-Global Alignment module is crucial for ensuring that local models effectively learn from the aggregated global information, preventing performance drops due to data heterogeneity.

Key Results

  • Superior Performance: EaDA consistently outperforms existing federated graph rationalization methods (like FedGR) and adapted centralized methods across all tested datasets, achieving higher accuracy (ACC) and AUC scores. For example, on Spurious-Motif with bias=0.9, EaDA achieves 0.5269 ± 0.0273, significantly better than Fed-vanillaGR's 0.3182 ± 0.0353.
  • Improved Rationale Extraction: On the Spurious-Motif dataset with ground-truth rationales, EaDA demonstrates significantly higher Precision@5 compared to baseline methods, indicating it extracts more accurate and faithful rationales.
  • Faster Training: EaDA is approximately 4.25 times faster than FedGR, making it more practical for real-world FL deployments.
  • Generalizability: Ablation studies show that both the ERE and LGA modules are critical for EaDA's performance. Furthermore, the ERE module itself can be integrated into other federated rationalization methods to improve their performance.
  • Scalability: EaDA shows good scalability, with performance degrading less than Fed-vanillaGR as the number of clients increases.

Limitations and Future Work

  • Hyperparameter Sensitivity: The paper notes that the performance of EaDA can be sensitive to the number of inferred environments (k). While they show optimal performance for certain k values, finding the ideal k might require tuning.
  • Computational Cost: While faster than FedGR, EaDA still involves additional computation for environment inference and counterfactual generation compared to a vanilla FL approach.
  • Future Work: The authors suggest exploring other dataset partitioning methods and investigating the structural generalizability of their ERE module further. They also imply that the framework could be extended to other graph-based FL tasks beyond rationalization.

Summary updated

Abstract

The success of Graph Neural Networks (GNNs) in graph classification has heightened interest in explainable GNNs, particularly through graph rationalization. This method aims to enhance GNNs explainability by identifying subgraph structures (i.e., rationales) that support model predictions. However, existing methods often rely on centralized datasets, posing challenges in scenarios where data privacy is crucial, such as in molecular property prediction. Federated Learning (FL) offers a solution by enabling collaborative model training without sharing raw data. In this context, Federated Graph Rationalization emerges as a promising research direction. However, in each client, the rationalization methods often rely on client-specific shortcuts to compose rationales and make task predictions. Data heterogeneity, characterized by non-IID data across clients, exacerbates this problem, leading to poor prediction performance. To address these challenges, we propose the Environment-aware Data Augmentation (EaDA) method for Federated Graph Rationalization. EaDA comprises two main components: the Environment-aware Rationale Extraction (ERE) module and the Local-Global Alignment (LGA) module. The ERE module employs prototype learning to infer and share abstract environment information across clients, which are then aggregated to form a global environment. This information is used to generate counterfactual samples for local clients, enhancing the robustness of task predictions. The LGA module uses contrastive learning methods to align local and global rationale representations, mitigating performance degradation due to data heterogeneity. Comprehensive experiments on benchmark datasets demonstrate the effectiveness of our approaches.
BibTeX
@inproceedings{yue2025empowering,
  author = {Linan Yue and Qi Liu and Yawen Li and Fangzhou Yao and Weibo Gao and Junping Du},
  title = {{Empowering Federated Graph Rationale Learning with Latent Environments}},
  booktitle = {The Web Conference},
  year = {2025},
  doi = {10.1145/3696410.3714929},
}
Figure