Task-Aware Adaptive Modulation: A Replay-Free and Resource-Efficient Approach For Continual Graph Learning

Jingtao Liu, Xinming Zhang

Continual Graph Learning (CGL) is the art of teaching a model to handle a never-ending stream of new graph data without it "forgetting" what it learned yesterday. Traditionally, researchers have relied on "replaying" old data or using massive, expensive pre-trained models. A new paper from researchers at the University of Science and Technology of China introduces Task-Aware Adaptive Modulation (TAAM), a method that achieves state-of-the-art performance with zero forgetting—all while keeping the core model frozen.

TL;DR

TAAM is a replay-free, resource-efficient approach to continual graph learning that uses lightweight "modulators" to steer the internal flow of a frozen GNN backbone. By isolating task-specific knowledge into these modules and using a prototype-guided strategy for retrieval, it achieves zero catastrophic forgetting and outperforms existing methods even when starting with a randomly initialized backbone.

What Problem Does This Paper Address?

In the world of machine learning, we face the Stability-Plasticity Dilemma. "Stability" is the ability to retain old knowledge, while "plasticity" is the ability to learn new things. Most models suffer from catastrophic forgetting: as they learn Task B, the weights shift so much that they completely lose the ability to perform Task A.

In Graph Class-Incremental Learning (GCIL), this is especially tricky because task identifiers (IDs) aren't available during inference. You might be classifying new types of molecules or evolving social communities, and the model has to "guess" which task context it's in. Current solutions usually fall into two buckets:

  1. Replay-based: Storing and re-training on old data. This is memory-intensive and raises privacy concerns.
  2. Prompt-based: Using "prompts" to guide a frozen model. These usually require a "pre-training tax"—they only work if you already have a massive, expensive pre-trained backbone.

Methods

The authors propose a "parameter-isolation" approach called Task-Aware Adaptive Modulation (TAAM). Instead of changing the whole network, they insert tiny, specialized modules into a frozen Graph Neural Network (GNN).

The Core Components:

  • Frozen Backbone: TAAM uses a standard GNN (like SGC) but keeps its weights fixed. Surprisingly, this works even if the backbone is randomly initialized.
  • Neural Synapse Modulators (NSM): These are the "steering wheels." For every task, a new NSM is created. It uses a node-attentive mechanism to generate specific scaling ($\gamma$) and shifting ($\beta$) factors (similar to FiLM layers) for every node. This allows for fine-grained control over how information flows through the GNN.
  • Prototype-Guided Strategy:
    • For Training: When a new task arrives, the model calculates a "prototype" (an average feature vector). It finds the most similar past task and copies its modulator's structure to give the new task a "warm start."
    • For Inference: Since task IDs are unknown at test time, the model calculates a prototype for the incoming test batch and selects the most relevant "expert" NSM from its bank.

Datasets

The researchers evaluated TAAM on six diverse and widely-used Graph Class-Incremental Learning benchmarks:

  • Citation Networks: Cora, Citeseer, CoraFull, and Arxiv.
  • Social Networks: Reddit.
  • Product Networks: Amazon Products.

They also tested the model under various "task division" protocols, including scenarios where the first task is much larger than subsequent ones, to simulate real-world imbalances.

Key Insights and Ideas

  • The "Pre-training Tax" is Optional: Most recent CGL breakthroughs rely on powerful pre-trained models. TAAM proves that you can achieve superior results by intelligently modulating a randomly initialized frozen backbone. This makes CGL accessible for domains where massive pre-trained graphs don't exist.
  • Modulation > Prompting: While "graph prompts" usually apply a single static change to an entire task, TAAM’s NSMs are node-attentive. They look at each node's specific features to decide how to modulate the signal, offering much higher expressivity.
  • Structural Isolation = Zero Forgetting: Because each task's knowledge is "locked" into its own frozen NSM, there is literally no way for new tasks to overwrite old ones. The "Average Forgetting" (AF) metric for TAAM is a perfect 0.0 across the board.

Key Results

TAAM established a new state-of-the-art across all six benchmarks:

  • Accuracy: On the large-scale Reddit dataset, TAAM achieved 99.5% accuracy, significantly outperforming the best replay-based method (96.9%). On Products, it hit 96.2% vs. the previous best of 73.5%.
  • Forgetting: While competitors like PDGNNs showed up to 12.7% forgetting on Arxiv, TAAM maintained 0.0% forgetting.
  • Efficiency: TAAM is significantly faster than replay-based methods because it doesn't need to re-process historical data. It also avoids the "Out-of-Memory" (OOM) errors that plague other methods on large base tasks.

Limitations and Future Work

While TAAM is highly efficient, the paper notes a few areas for growth:

  • Batch Dependency: The inference strategy relies on calculating a prototype for a batch of test nodes to identify the task. If the model were asked to classify a single, isolated node without any context, it might struggle to select the correct modulator.
  • Linear Scaling: The "bank" of modulators grows with the number of tasks. While these modules are very lightweight, a scenario with thousands of tiny tasks might eventually lead to a large parameter overhead.
  • Complex Scenarios: The authors suggest that future work should explore even more challenging "task-agnostic" settings where class boundaries are even more blurred.

Paper Title: Task-Aware Adaptive Modulation: A Replay-Free and Resource-Efficient Approach For Continual Graph Learning Authors: Jingtao Liu and Xinming Zhang.

Summary updated

Abstract

Continual Graph Learning(CGL)focuses on acquiring new knowledge while retaining previously learned information, essential for real-world graph applications. Current methods grapple with two main issues:1) The Stability-Plasticity Dilemma: Replay-based methods often create an imbalance between the Dilemma, while incurring significant storage costs.2) The Resource-Heavy Pre-training: Leading replay-free methods critically depend on extensively pre-trained backbones, this reliance imposes a substantial resource burden.In this paper, we argue that the key to overcoming these challenges lies not in replaying data or fine-tuning the entire network, but in dynamically modulating the internal computational flow of a frozen backbone. We posit that lightweight, task-specific modules can effectively steer a GNN's reasoning process. Motivated by this insight, we propose Task-Aware Adaptive Modulation(TAAM), a replay-free, resource-efficient approach that charts a new path for navigating the stability-plasticity dilemma. TAAM's core is its Neural Synapse Modulators(NSM), which are trained and then frozen for each task to store expert knowledge. A pivotal prototype-guided strategy governs these modulators: 1) For training, it initializes a new NSM by deep-copying from a similar past modulator to boost knowledge transfer. 2) For inference, it selects the most relevant frozen NSM for each task. These NSMs insert into a frozen GNN backbone to perform fine-grained, node-attentive modulation of its internal flow-different from the static perturbations of prior methods. Extensive experiments show that TAAM comprehensively outperforms state-of-the-art methods across six GCIL benchmark datasets. The code will be released upon acceptance of the paper.
BibTeX
@misc{liu2025task,
  author = {Jingtao Liu and Xinming Zhang},
  title = {{Task-Aware Adaptive Modulation: A Replay-Free and Resource-Efficient Approach For Continual Graph Learning}},
  year = {2025},
  eprint = {2509.00735},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2509.00735},
}
Figure