Cross-Task Architectural Transferability in Graph Learning

78 concepts78 papers1 proposal

Graph Neural Networks (GNNs) are typically evaluated on a single downstream task—most commonly, semi-supervised node classification. However, graph data inherently supports multiple levels of inference: node-level (e.g., predicting a user's interests), edge-level (e.g., predicting whether two users will connect), and graph-level (e.g., predicting the toxicity of a molecule).

A major open challenge in graph representation learning is cross-task architectural transferability. When an architecture, data condensation method, or self-supervised objective is optimized for one scale of inference, it develops a task-specific bias. This bias often prevents the learned representations or compressed structures from generalizing to other graph-based objectives.

The Divergent Informational Needs of Graph Tasks

To understand why cross-task generalization fails, we must look at what different tasks require from a graph's topology and feature space:

  • Node Classification relies heavily on local homophily (neighbors sharing similar traits) and feature smoothing.
  • Link Prediction requires precise pairwise structural metrics, such as shortest-path distances or asymmetric directed edge properties.
  • Graph Classification demands global topological invariants and macro-scale structural patterns.

Because these requirements conflict, optimizations that excel at one level can actively harm performance at another. For instance, Abu-El-Haija et al. demonstrate in Learning Edge Representations via Low-Rank Asymmetric Projections that modeling asymmetric directed edges specifically for link prediction does not inherently translate to node-level objectives. Conversely, deep attention mechanisms like AERO-GNN, proposed by Lee et al. in Towards Deep Attention in Graph Neural Networks: Problems and Remedies, refine node features effectively but remain unproven for edge-level or global graph-level classification.

The "One-Trick Pony" of Graph Condensation

The cross-task generalization gap is particularly severe in graph condensation, where a massive graph is distilled into a tiny synthetic proxy dataset.

Early frameworks like GCOND, introduced by Jin et al. in Graph Condensation for Graph Neural Networks, synthesize data by matching the training gradients of a GNN on the original graph. Because these gradients are derived from node classification, the resulting proxy graphs discard the structural information necessary for link prediction or multi-label settings. As highlighted by Qiu et al. in their large-scale benchmark GCondenser: Benchmarking Graph Condensation, condensed graphs optimized for a single task rarely support diverse downstream objectives.

To break this task-dependency, Chen et al. proposed Contrastive Graph Condensation: Advancing Data Versatility through Self-Supervised Learning. They argue that traditional condensation methods suffer from a severe task-specific bias. By shifting to a self-supervised framework that disentangles structure from attributes, they produce versatile condensed graphs capable of serving classification, link prediction, and clustering simultaneously. Without such explicit multi-task design, acceleration frameworks like the precomputation method developed by Li et al. in Adapting Precomputed Features for Efficient Graph Condensation remain structurally bound to their initial node classification objectives.

Task-Specific Scope in Knowledge Distillation

Knowledge distillation (KD) compresses large GNNs into lightweight students or Multi-Layer Perceptrons (MLPs). However, the structural knowledge transferred during distillation is often bounded by the scope of the teacher's training task.

For example, Wu et al. introduce an adversarial KD framework in Compressing Deep Graph Neural Networks via Adversarial Knowledge Distillation, but note that its validation is restricted to discrete classification, leaving its transferability to continuous regression or edge prediction uncertain. Similarly, the SA-MLP framework by Chen et al. (SA-MLP: Distilling Graph Knowledge from GNNs into Structure-Aware MLP) and hypergraph distillation methods like DistillHGNN (DistillHGNN: A Knowledge Distillation Approach for High-Speed Hypergraph Neural Networks) successfully eliminate message-passing overhead for node classification, but their architectural suitability for edge-level reasoning remains an open question.

When the target task shifts, the underlying structural assumptions must shift too. For instance, path-based reasoning in knowledge graphs requires distance-based message pruning, as shown by Shomer et al. in Distance-Based Propagation for Efficient Knowledge Graph Reasoning, a mechanism that does not directly translate to standard node classification. Conversely, Ahmad Sajedi designs a structural attention-matching technique for graph classification in GSTAM: Efficient Graph Distillation with Structural Attention-Matching, but notes that adapting it to node-level classification requires addressing entirely different structural dependencies.

The Node-to-Graph Gap in Self-Supervised Learning

Graph Contrastive Learning (GCL) attempts to learn general-purpose representations by maximizing mutual information between different augmented views of a graph. Yet, even unsupervised representations fall into the cross-task generalization trap.

Many GCL frameworks optimize for node-to-neighborhood consistency. As Dong et al. show in Node Representation Learning in Graph via Node-to-Neighbourhood Mutual Information Maximization, maximizing this local mutual information yields excellent node classifiers but does not guarantee expressive global graph-level representations. Similarly, spectral contrastive frameworks like PolyGCL (PolyGCL: GRAPH CONTRASTIVE LEARNING via Learnable Spectral Polynomial Filters) and LOHA (LOHA: Direct Graph Spectral Contrastive Learning Between Low-Pass and High-Pass Views) decouple low-pass and high-pass graph frequencies to handle homophily and heterophily. While these spectral decoupling strategies are highly effective for node-level tasks, it remains uncertain whether they capture the topological properties required for link prediction or whole-graph regression.

Data augmentation strategies face the same bottleneck. Adaptive augmentations like GCA, designed by Zhu et al. (Graph Contrastive Learning with Adaptive Augmentation), protect critical local structural hubs. While this preserves the semantic identity of individual nodes, the authors leave it uncertain whether preserving local hubs is sufficient for tasks where the global identity of the entire graph is the primary objective.

Implications for Federated and Distributed Environments

The cross-task generalization gap compounds when moving to distributed or federated settings, where data cannot be centrally pooled.

In Federated Graph Learning (FGL), clients often share condensed background data or calibrate local models to mitigate domain drift. For example, the FedBG framework by Deng et al. (FedBG: Proactively Mitigating Bias in Cross-Domain Graph Federated Learning Using Background Data) and the FGSSL framework by Huang et al. (Federated Graph Semantic and Structural Learning) calibrate node-level semantics and local structures. However, because these calibrations are tailored for node classification, they fail to capture the global topology required for graph-level tasks like molecular property prediction, where the entire graph is the primary unit of analysis.

Ultimately, whether compressing graphs, distilling models, or designing self-supervised objectives, assuming that success on node classification guarantees transferability to edge- or graph-level objectives is a fundamental misstep. Bridging this cross-task gap requires moving away from single-task optimization and explicitly designing architectures that preserve multi-scale structural properties.

Key Foundational Work

Learning Edge Representations via Low-Rank Asymmetric Projections (2017) — contributed 1 concept elements to this direction.
Stability and Generalization of Graph Convolutional Neural Networks (2019) — contributed 1 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_cross_task_generalization_cross_task_arc_0c0227f5,
  title = {cross-task generalization (cross-task architectural transferability)},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/cross-task-generalization-cross-task-architectural-transferability/rdr_0c0227f5f1804b49b2388e6fc8d2e148},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-21},
}

This exact version (version 1, frozen text):

@misc{lacuna_cross_task_generalization_cross_task_arc_0c0227f5_v1,
  title = {cross-task generalization (cross-task architectural transferability)},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/cross-task-generalization-cross-task-architectural-transferability/rdr_0c0227f5f1804b49b2388e6fc8d2e148/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-21},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake