TL;DR
This paper introduces EaDA, a novel federated learning framework for graph rationalization that tackles data heterogeneity and local shortcut learning. By inferring and leveraging "latent environments" across clients, EaDA generates more robust rationales, significantly improving prediction performance in privacy-preserving settings.
What Problem Does This Paper Address?
Graph Neural Networks (GNNs) are powerful for graph classification, but understanding why they make a prediction is crucial. Graph rationalization aims to identify the specific subgraph (the "rationale") that supports a GNN's decision. However, many existing rationalization methods rely on centralized datasets, which is problematic for sensitive data like molecular properties where privacy is paramount. Federated Learning (FL) offers a solution by allowing collaborative training without sharing raw data.
The challenge in Federated Graph Rationalization is data heterogeneity. Different clients in an FL system often have non-independent and identically distributed (non-IID) data, meaning their data distributions vary. This heterogeneity exacerbates a common issue in rationalization: models exploiting "shortcuts" – spurious correlations in the data that help make predictions but aren't causally linked to the true outcome. These shortcuts are often environment-specific, meaning they differ across clients, leading to poor global model performance. Previous attempts to address this, like FedGR, are computationally expensive.
Methods
The proposed solution, Environment-aware Data Augmentation (EaDA), tackles these issues with two core components:
- Environment-aware Rationale Extraction (ERE):
- Latent Environment Inference: Instead of assuming environments are unavailable (like FedGR), EaDA infers them. It uses prototype learning on the complement subgraphs (nodes not part of the rationale) to capture environment-specific features. These inferred "environments" are then aggregated on the server to form a global environment.
- Counterfactual Data Generation: Clients use this global environment information to generate counterfactual samples. This involves transforming local rationale representations into different environments, creating new data points that disrupt spurious correlations and make the model more robust. The labels of these generated samples remain unchanged.
- Local-Global Alignment (LGA):
- Contrastive Learning: This module uses contrastive learning to align the local rationale representations with the global rationale representations. This helps the local models learn from the global information, further mitigating the performance degradation caused by data heterogeneity.
The overall training process involves clients performing ERE and LGA, then sending updated models and inferred environments to the server for aggregation.
Datasets
The paper evaluates EaDA on:
- Synthetic Dataset: Spurious-Motif, a benchmark designed to introduce shortcuts by varying the relationship between motif subgraphs (rationales) and base subgraphs (complements). Datasets with different levels of bias (b=0.5, 0.7, 0.9) were used.
- Real-world Datasets: Several datasets from the Open Graph Benchmark (OGB), specifically MolHIV, MolToxCast, MolBBBP, MolBACE, and MolSIDER. These datasets are commonly used for molecular property prediction.
For FL scenarios, the authors employ a Latent Dirichlet Allocation (LDA) based partitioning to create heterogeneous data distributions across clients.
Key Insights and Ideas
- Inferring Environments is Key: The core insight is that instead of assuming environments are unknown or trying to perfectly reconstruct them, inferring abstract "latent environments" is sufficient and more practical for federated graph rationalization.
- Complement Subgraphs as Environment Indicators: The paper cleverly uses the complement subgraph (non-rationale parts) to infer the environment. This is because these parts are more likely to capture variations and discriminative features related to the data distribution.
- Counterfactuals for Robustness: Generating counterfactual data by transforming local rationales into different inferred environments is a powerful way to break spurious correlations and improve the faithfulness of extracted rationales.
- LGA Bridges the Gap: The Local-Global Alignment module is crucial for ensuring that local models effectively learn from the aggregated global information, preventing performance drops due to data heterogeneity.
Key Results
- Superior Performance: EaDA consistently outperforms existing federated graph rationalization methods (like FedGR) and adapted centralized methods across all tested datasets, achieving higher accuracy (ACC) and AUC scores. For example, on Spurious-Motif with bias=0.9, EaDA achieves 0.5269 ± 0.0273, significantly better than Fed-vanillaGR's 0.3182 ± 0.0353.
- Improved Rationale Extraction: On the Spurious-Motif dataset with ground-truth rationales, EaDA demonstrates significantly higher Precision@5 compared to baseline methods, indicating it extracts more accurate and faithful rationales.
- Faster Training: EaDA is approximately 4.25 times faster than FedGR, making it more practical for real-world FL deployments.
- Generalizability: Ablation studies show that both the ERE and LGA modules are critical for EaDA's performance. Furthermore, the ERE module itself can be integrated into other federated rationalization methods to improve their performance.
- Scalability: EaDA shows good scalability, with performance degrading less than Fed-vanillaGR as the number of clients increases.
Limitations and Future Work
- Hyperparameter Sensitivity: The paper notes that the performance of EaDA can be sensitive to the number of inferred environments (k). While they show optimal performance for certain k values, finding the ideal k might require tuning.
- Computational Cost: While faster than FedGR, EaDA still involves additional computation for environment inference and counterfactual generation compared to a vanilla FL approach.
- Future Work: The authors suggest exploring other dataset partitioning methods and investigating the structural generalizability of their ERE module further. They also imply that the framework could be extended to other graph-based FL tasks beyond rationalization.
Summary updated