Abstract
Deep Graph Reprogramming (GARE) introduces a novel paradigm for reusing pre-trained Graph Neural Networks (GNNs) across disparate domains without updating model parameters. By employing Data Reprogramming (DARE) and Model Reprogramming (MERE), GARE enables a single frozen model to perform tasks ranging from citation network classification to molecular property prediction. Our evaluation across 14 diverse datasets demonstrates that GARE achieves performance competitive with models trained from scratch while significantly reducing computational and storage overhead.
Introduction
Graph Neural Networks (GNNs) have emerged as powerful tools for modeling non-Euclidean data. However, their practical deployment is often hindered by the "irregularity" of graph data; unlike image models that accept standard RGB inputs, GNNs must handle varying feature dimensions and topological structures. Currently, adapting a GNN to a new task typically requires Knowledge Distillation or full fine-tuning, both of which necessitate training and storing a new model for every specific application. This overhead is particularly prohibitive for resource-constrained edge devices.
This paper addresses the challenge: Can we utilize a single pre-trained, frozen GNN to solve multiple downstream tasks across different domains? We propose Deep Graph Reprogramming (GARE), a framework that "reprograms" the input data and the model's internal logic to bridge the gap between pre-training and downstream objectives.
Methodology
GARE decomposes the reprogramming challenge into two primary paradigms: Data Reprogramming (DARE) and Model Reprogramming (MERE).
Data Reprogramming (DARE)
DARE focuses on aligning heterogeneous input graphs with the expectations of the frozen GNN. It consists of three key techniques:
- Meta-FeatPadding (MetaFP): To handle feature dimension mismatches (e.g., mapping 10-dimensional features to a 100-dimensional input layer), MetaFP learns task-aware perturbations. These perturbations act as "guards" that trick the frozen GNN into processing the data correctly.
- Edge-Slimming (EdgSlim): For tasks where the graph structure itself hinders performance, EdgSlim utilizes gradient information to identify and remove detrimental edges.
- Meta-GraPadding (MetaGP): In inductive settings where new graphs are encountered at test time, a small "meta-graph" is appended to the input to guide the frozen model's predictions.
Model Reprogramming (MERE)
MERE focuses on the model's internal behavior, specifically the neighbor aggregation logic. Different domains often benefit from different aggregation strategies (e.g., Mean vs. Max). We introduce the Reprogrammable-Aggregator (ReAgg), which uses the Gumbel-Max trick to dynamically select the optimal aggregation method for a specific task. While the aggregator choice is learned, the underlying GNN weights remain strictly frozen.
Experimental Results
We evaluated GARE on 14 diverse datasets spanning citation networks, e-commerce, biochemistry, 3D objects, and action recognition.
Performance and Efficiency
GARE demonstrates remarkable cross-domain success. For instance, a model pre-trained on product category prediction was successfully repurposed for molecule regression, reducing the Mean Absolute Error (MAE) from 24.18 (vanilla) to 2.30. On the Citeseer dataset, GARE achieved 92.29% accuracy on the "Computers" task, approaching the 94.75% achieved by a model trained from scratch.
The Impact of Aggregation
A key insight of this work is that neighbor aggregation is a critical lever for cross-domain transfer. As shown in the figure below, simply varying the aggregation function can lead to significant fluctuations in downstream accuracy, justifying the need for the ReAgg component in MERE.
Discussion and Conclusion
GARE represents a shift from traditional fine-tuning toward a more efficient "reprogramming" approach. By turning adversarial-style perturbations into assets (MetaFP) and dynamically selecting aggregation logic, we can reuse large-scale pre-trained GNNs with minimal overhead. While GARE may slightly underperform compared to fine-tuning when the domain gap is very small, its strength lies in its ability to bridge large domain gaps (e.g., citation to molecule) with extremely fast convergence.
Future work will explore generalizing this framework to more complex graph tasks and investigating additional dynamic network strategies beyond neighbor aggregation.
Summary updated