Introduction
Continual Graph Learning (CGL) is a burgeoning field aimed at enabling Graph Neural Networks (GNNs) to learn from a stream of tasks sequentially. Unlike static learning, CGL requires models to accumulate knowledge without suffering from "catastrophic forgetting" (CF)—the tendency of neural networks to lose performance on previously learned tasks when exposed to new data. While significant progress has been made in node-level CGL, graph-level tasks, such as molecular property prediction, present unique challenges due to the complex structural dependencies and the presence of spurious correlations.
The Challenge of Spurious Correlations
A primary driver of catastrophic forgetting in graph-level tasks is the model's reliance on "shortcuts" or spurious correlations. During training on a specific task, a model may associate a label with background noise—termed the "environment"—rather than the underlying causal features, or the "rationale."
For example, in chemical datasets, a model might incorrectly link a molecule's activity to a common but irrelevant side-chain present in the initial training set. When subsequent tasks introduce different molecular contexts, these learned shortcuts become detrimental, leading to a significant drop in accuracy on historical tasks.
The RL-GNN Framework
To address these issues, the authors of the AAAI-25 paper "Exploring Rationale Learning for Continual Graph Learning" propose RL-GNN (Rationale Learning Graph Neural Network). This framework treats CGL as an invariant learning problem, focusing on identifying stable structural patterns that remain valid across varying environments.
Rationale-Environment Decoupling
RL-GNN utilizes a rationale generator to produce a soft mask for each graph, effectively partitioning it into a rationale (the core informative subset) and an environment (the remaining noise). By isolating the rationale, the model can focus on features that are intrinsically linked to the target label.
Virtual Environment Construction
A key innovation in RL-GNN is the use of "virtual environment swapping." To simulate potential distribution shifts, the model pairs the rationale of one graph with the environment of another within the same mini-batch. This creates diverse counterfactual views, forcing the model to learn representations that are invariant to the surrounding noise.
Contrastive Invariant Learning
The training process involves a supervised contrastive objective. This objective encourages the graph encoder to produce consistent representations for a given rationale, regardless of the virtual environment it is embedded in. This approach mitigates the "recency bias" often found in standard cross-entropy loss, which tends to over-optimize for the current task's distribution.
Experimental Evaluation
The effectiveness of RL-GNN was evaluated on three real-world benchmarks: Aromaticity-CL, REDDIT-MULTI-12K-CL, and ENZYMES-CL. These datasets provide long task sequences and diverse structural data, offering a rigorous test for CGL methods.
RL-GNN demonstrated significant improvements over state-of-the-art baselines:
- Superior Accuracy: On the 15-task Aromaticity-CL dataset, RL-GNN achieved an Average Performance (AP) of 68.24%, nearly doubling the performance of the Experience Replay (ER-GNN) baseline (37.54%).
- Minimal Forgetting: The model exhibited remarkably low Average Forgetting (AF). On Aromaticity, RL-GNN's forgetting was limited to -4.95%, whereas traditional regularization methods like EWC saw drops exceeding -60%.
Conclusion
RL-GNN represents a significant advancement in graph-level continual learning by shifting the focus from simple data rehearsal to the discovery of invariant rationales. By decoupling causal features from environmental noise and utilizing virtual environment interventions, the framework provides a robust defense against catastrophic forgetting. This "rationale-first" approach holds great promise for complex domains like drug discovery and social network analysis, where identifying stable, underlying patterns is essential for long-term model reliability.
Summary updated