Topology Preserving Continual Graph Learning

21 concepts12 papers1 proposal

When a machine learning model learns a sequence of tasks, it typically suffers from catastrophic forgetting—overwriting past knowledge to accommodate new data. In the image domain, standard continual learning strategies mitigate this by saving a buffer of past images (experience replay) or penalizing changes to important weights (regularization). However, as Carta et al. demonstrated in a systematic benchmark (Catastrophic Forgetting in Deep Graph Networks: A Graph Classification Benchmark), these standard "flat" approaches fail when applied to Graph Neural Networks (GNNs).

The failure stems from a fundamental difference in data geometry. In an image, pixels sit on a fixed grid. In a graph, a node's identity and meaning are defined by its relationships with its neighbors. GNNs capture this through message passing, where node representations are iteratively aggregated across edges. If you store isolated nodes for replay, or regularize weights without considering how they affect message propagation, the model loses the structural context—the receptive field—needed to reconstruct past knowledge.

To solve this, a prominent research direction has emerged: Topology-Preserving Continual Graph Learning. The core intuition is that to prevent forgetting in GNNs, models must explicitly protect and replay the relational skeleton of the graph, rather than just individual node features or raw task outputs.

Topology-Aware Regularization: Protecting the Propagation Pathway

One way to prevent forgetting is to restrict changes to the model parameters that matter most for past tasks. In graphs, the most critical parameters are those that govern how information flows across neighborhoods.

Liu et al. introduced this concept with Topology-aware Weight Preserving (TWP) in Overcoming Catastrophic Forgetting in Graph Neural Networks. Instead of calculating parameter importance based solely on the final task loss, TWP explicitly measures how sensitive the local topological structures and neighborhood aggregation strategies are to changes in the weights. By penalizing changes to parameters that significantly alter edge-based information propagation, the model locks in its structural understanding.

As surveyed by Febrinanto et al. in Graph Lifelong Learning: A Survey, this topology-aware regularization ensures that incremental updates do not overwrite the structural integrity of existing relationships. This insight is reinforced by Zhang et al. in CGLB: Benchmark Tasks for Continual Graph Learning, who showed that protecting the topological importance of key nodes and edges stabilizes the model's understanding of global topology far better than standard weight regularization.

Structural Rehearsal: Replaying Subgraphs and Coarsened Motifs

When memory buffers are available, replaying raw, disconnected data points is insufficient because message passing requires edges. Therefore, effective rehearsal requires saving or reconstructing structural sub-units.

Isolated Replay (Fails):     [Node A]   [Node B]   [Node C]  --> No structural context

Structural Rehearsal:        [Node A] ——— [Node B] ——— [Node C]  --> Preserves message-passing pathways

For evolving or temporal networks, Feng et al. argued in Towards Open Temporal Graph Neural Networks that the network's fundamental evolutionary patterns are stored in its local geometry. By replaying fundamental three-node motifs (triads) rather than isolated nodes, the model retains the precise rules of how connections formed in older data distributions.

When dealing with large-scale graphs, storing entire subgraphs becomes memory-prohibitive. To circumvent this, researchers use graph compression techniques that preserve connectivity:

Global Alignment and Distillation: Preventing Structural Drift

Beyond local neighborhoods, continual learning causes "structural drift"—a phenomenon where the global coordinate space or relative distances between distant node representations distort over time.

To combat global drift, Hoang et al. developed Universal Graph Continual Learning (UGCL) in Universal Graph Continual Learning. The authors combined experience replay with explicit local and global structure distillation. By forcing the current model's layer-wise representations to maintain the same relative topological distances as the historical model, they achieved a synergistic effect that outperforms data rehearsal alone.

Taking global alignment further, Zhang et al. introduced SAOT (SAOT: Self-Supervised Continual Graph Learning with Structure-Aware Optimal Transport). SAOT utilizes Optimal Transport theory to mathematically align the global topology of the graph with its representation space, ensuring that the macro-level map of node relationships remains stable across long sequences of tasks.

Even when the network structure explicitly changes via edge deletions, the structural history must be accounted for. Li et al. demonstrated in Joint Learning of Evolving Links for Dynamic Network Embedding that maintaining a dual-graph structural memory—one tracking active connections and another preserving deleted links—prevents the loss of historical context and stabilizes node representations over time.

Summary

Topology-preserving continual graph learning works because it aligns with the core mechanism of graph neural networks. GNNs understand the world through connectivity. By designing regularization terms that protect propagation weights, compressing graphs into topology-preserving coarsened structures, and using distillation to prevent global structural drift, these methods ensure that the model's structural memory remains intact across a continuous stream of information.

Key Foundational Work

Overcoming Catastrophic Forgetting in Graph Neural Networks (2021) — contributed 2 concept elements to this direction.
Joint Learning of Evolving Links for Dynamic Network Embedding (2018) — contributed 1 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_structural_motif_replay_d76f1f5f,
  title = {structural motif replay},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/structural-motif-replay/rdr_d76f1f5f9d8944d3861432c75718ae20},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_structural_motif_replay_d76f1f5f_v1,
  title = {structural motif replay},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/structural-motif-replay/rdr_d76f1f5f9d8944d3861432c75718ae20/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake