Most Graph Neural Networks (GNNs) are trained under a "static world" assumption: the nodes, edges, and classes are fixed during training and deployment. In reality, graphs are living structures. Social networks gain new users, financial systems see new types of fraud, and citation networks evolve with new research fields.
When a standard GNN is updated with new data, it suffers from catastrophic forgetting. The optimization process for new information overwrites the weights and representations learned from previous tasks, causing performance on historical data to plummet. Xikun Zhang and colleagues note that this decline is particularly sharp in graphs because the model must maintain not just node features, but the underlying structural patterns that connect them (Continual Learning on Graphs: Challenges, Solutions, and Opportunities).
Continual Graph Learning (CGL) aims to bridge this gap, allowing models to learn incrementally while maintaining a "long-term memory" of the graph's evolution.
The Challenge of Relational Context
In traditional image-based continual learning, samples are independent. In graphs, nodes are defined by their neighbors. If you save a "representative" node to remember a past task but discard its neighbors, you lose the structural context that made that node informative.
To solve this, researchers like Fan Zhou and Chengtai Cao pioneered Experience Replay (ER) for graphs. Their framework, ER-GNN, stores a small buffer of representative nodes from previous tasks and "replays" them during new training sessions to anchor the model’s memory.
However, simply picking random nodes is inefficient. Recent work has focused on topology-aware selection:
- Feature-Topology Fusion: Jinhui Pang and Changqing Lin proposed FTF-ER, which uses a "Hodge Potential Score" to rank nodes based on their global importance in the graph's flow, ensuring the most structurally significant nodes are preserved.
- Structural Entropy: Tianchi Xie introduced SES, which uses Shapley values to decompose a graph's structural entropy. By selecting samples that preserve the "global organization" of the dataset, they can maintain performance with far fewer stored samples.
Prototypes as Semantic Anchors
Instead of storing raw nodes, which can be memory-intensive and raise privacy concerns, another direction uses Prototypes. A prototype is a condensed representation (often a mean vector) of a class or a task in the embedding space.
Bin Lu and Xiaoying Gan developed Geometer, which represents node categories as dynamic, attention-based prototypes. When new classes emerge, the model maps them into a geometric space that respects the boundaries of old classes. Similarly, Xikun Zhang proposed Hierarchical Prototype Networks, which use a three-level hierarchy of abstractions to refine only the relevant parts of the model for new tasks, keeping memory consumption strictly bounded.
Plasticity through Parameter Isolation
A third strategy avoids the "overwriting" problem by physically or logically separating the parameters used for different tasks.
Peiyan Zhang introduced PI-GNN, which identifies and "freezes" the parameters critical for stable parts of the graph while expanding the network to accommodate new data. This ensures that the "stability" of old knowledge is never compromised by the "plasticity" required for new learning.
For massive knowledge graphs, Jiajun Liu and Peng Wang developed FastKGE. They leverage Low-Rank Adaptation (LoRA) to isolate new knowledge into incremental "adapters." This allows the model to learn new facts by adding tiny, efficient layers rather than retraining the entire embedding space.
Condensation and Hallucination
When the graph grows too large for simple replay, researchers turn to Graph Condensation. The goal is to create a tiny, synthetic graph that carries the same mathematical "signal" as the original massive dataset.
Ruihong Qiu and Hongzhi Yin proposed PUMA, which condenses incoming graph streams into highly informative "pseudo-labels." In a similar vein, the DMSG framework goes a step further by using generative models to "hallucinate" new variations of historical nodes, ensuring the replay buffer remains diverse and robust against distribution shifts.
The Future: Decentralization and Foundation Models
The field is now moving toward two frontiers:
- Federated Continual Learning: Yinlin Zhu and Xunkai Li are exploring how to perform these updates across decentralized devices (like mobile phones) without sharing raw private data, using "pseudo-prototypes" to sync knowledge (Federated Continual Graph Learning).
- Graph Foundation Models: Researchers are investigating whether Large Language Models (LLMs) can act as a "universal backbone" for graphs. GraphGPT suggests that by aligning graph structures with natural language instructions, we can create models that generalize to new domains without the catastrophic forgetting seen in specialized GNNs.