TL;DR
SA²GFM is a new framework designed to make Graph Foundation Models (GFMs) significantly more robust against noise and adversarial attacks. By injecting hierarchical structural information via textual prompts and using a "null expert" to filter out irrelevant cross-domain knowledge, it achieves state-of-the-art performance even when graphs are heavily corrupted.
What Problem Does This Paper Address?
Graph Foundation Models (GFMs) aim to be the "GPT of graphs," learning general representations across multiple domains that can be easily adapted to specific tasks. However, they face two major hurdles:
- Fragility: Most GFMs use shallow Graph Neural Network (GNN) backbones that are theoretically limited by the 1-Weisfeiler-Lehman (1-WL) test. This means they struggle to distinguish complex structural patterns and are highly vulnerable to random noise or targeted adversarial attacks (like adding or removing specific edges).
- Negative Transfer: When adapting a model from one domain (e.g., citation networks) to another (e.g., product co-purchase graphs), the differences in structure can actually degrade performance if the model tries to force-fit irrelevant knowledge.
Methods
The authors propose SA²GFM (Structure-Aware Semantic Augmentation GFM), which tackles these issues through three main components:
- Structure-Aware Semantic Augmentation: The model uses graph structural entropy to build a hierarchical "encoding tree." This tree is converted into natural language prompts (e.g., "Node 1 belongs to cluster A which contains 2 nodes..."). These prompts are processed by a language model (BERT) to enrich the raw node features with high-level structural context.
- Self-Supervised Information Bottleneck (SS-IB): To ensure the model doesn't just memorize noise, the authors implement an Information Bottleneck objective. This forces the model to compress redundant input information while maximizing the "signal" needed for downstream tasks.
- Expert Adaptive Routing with a Null Expert: During adaptation, the model uses a Mixture-of-Experts (MoE) architecture. Crucially, it includes a null expert—a shallow GCN trained only on the target graph. If the pre-trained experts from other domains are a poor fit, the router assigns more weight to the null expert, effectively "rejecting" harmful knowledge transfer.
Datasets
The model was evaluated across seven benchmark datasets spanning three distinct domains:
- Citation Domain: Cora, CiteSeer, PubMed, and ogbn-arXiv.
- Products Domain: ogbn-Tech and ogbn-Home.
- Web Page Domain: Wiki-CS.
Evaluation focused on two challenging scenarios: Cross-Dataset (same domain, different dataset) and Cross-Domain (entirely different fields).
Key Insights and Ideas
- Text as a Structural Bridge: One of the most creative ideas is using LLMs to bridge the gap between structure and semantics. By describing a node's structural role in words, the model leverages the rich semantic space of pre-trained language models to understand graph topology.
- The Power of "No": The "null expert" is a simple but powerful solution to the negative transfer problem. It acknowledges that sometimes, the best thing a foundation model can do is realize it doesn't know enough about a new domain and rely on local information instead.
- Hierarchical Refinement: Unlike many models that treat graph structure as a flat set of edges, SA²GFM optimizes the graph at two levels during fine-tuning: intra-cluster (local fidelity) and inter-cluster (global signal propagation).
Key Results
SA²GFM was compared against 9 state-of-the-art baselines and consistently came out on top:
- Superior Robustness: On average, it achieved a +5.9% accuracy improvement in node classification and +2.4% in graph classification over the best-performing runner-up.
- Stability Under Attack: As the intensity of noise or adversarial perturbations increased, SA²GFM's performance declined much more slowly than its competitors. In severe noise settings ($\lambda = 0.8$), it maintained a significant lead.
- Cross-Domain Success: In challenging cross-domain transfers, it achieved a +5.1% average gain over MDGFM, the strongest existing robust baseline.
Limitations and Future Work
While robust, the authors note several areas for improvement:
- Complex Attacks: The current evaluation focuses on standard perturbations. Future work could test resilience against more sophisticated, multi-step topological attacks.
- Graph Diversity: The framework has primarily been validated on homogeneous, static graphs. Extending SA²GFM to heterogeneous (multiple node/edge types) or dynamic (time-varying) graphs is a logical next step.
- Computational Cost: While the authors argue the structure optimization is efficient, the use of BERT for feature augmentation adds an inference overhead that might be challenging for extremely large-scale real-time applications.
Summary updated