Preprocessing Overhead in Scalable Graph Neural Networks

You are reading version 1 of this direction, frozen 2026-10-04. The latest version is version 2.

Version 1 of 2278 papers300 conceptsfrozen 2026-10-04

Graph Neural Networks (GNNs) face a fundamental scalability paradox. To process massive datasets like social networks or biological interactomes, researchers have developed a suite of techniques to shrink graphs, decouple computations, or simplify neighborhoods. However, these efficiency gains often come with a "hidden tax": a significant preprocessing phase that can be as computationally expensive as the training itself.

The Scalability Paradox

The standard GNN training loop is notoriously memory-intensive because a node’s representation depends on an exponentially growing neighborhood. To solve this, Zengfeng Huang and colleagues in Scaling Up Graph Neural Networks Via Graph Coarsening proposed a "train small, predict big" strategy. By coarsening a graph by up to 10x, they could train on a tiny proxy and apply the weights back to the original structure.

While this saves GPU memory, the initial computation required to partition or coarsen the graph often shifts the bottleneck from the GPU training loop to CPU-based graph processing. As Shuyin Xia and Xinjun Ma note in their work on Supervised Granular-Ball Graph Coarsening, these advanced reduction techniques require intensive optimization phases before the primary model training can even begin.

The Circular Efficiency Problem

A recurring challenge in this field is what Charles Dickens and Jiong Zhu describe as a "circular efficiency problem" in Graph Coarsening via Convolution Matching. Many graph condensation methods require training a model on the full-scale graph first to generate a reduced summary or to match gradients. This requirement defeats the primary goal of data reduction: if you must process the whole graph to simplify it, you haven't truly bypassed the original computational burden.

To break this cycle, Mridul Gupta and Sayan Ranu introduced Bonsai, a gradient-free condensation method. By focusing on the geometry of "computation trees" rather than matching training gradients, they achieved significant speedups without needing a full-dataset training pass.

Decoupling and Structural Reformulation

Another major direction involves "decoupling" the expensive graph propagation from the learnable transformations. Models like LD2 by Ningyi Liao and ScaDyG by Xiang Wu precompute structural influences or node embeddings. Once these are stored, the training phase becomes lightning-fast.

However, this "one-time" cost is not trivial. For million-scale graphs, pre-calculating multi-hop aggregations—as seen in NAGphormer by Jinsong Chen—or mapping neighborhoods into structured grids—as in Node2Grids by Chuan Chen—can take hours or even days. This latency is particularly problematic for dynamic graphs where the structure changes frequently, necessitating a full re-calculation of the input features.

Measuring the True Cost of Learning

The research community is increasingly calling for "end-to-end latency" evaluation. Mohammad Matin Najafi and the creators of the BEACON benchmark argue that the perceived speed advantages of machine learning models for graph tasks are often overstated. When the time for feature extraction, graph partitioning, and data preparation is factored in, traditional combinatorial algorithms can sometimes outperform neural approaches in real-world applications.

The shift toward more efficient preprocessing—such as the GPU-initiated storage access proposed by Jeongmin Brian Park in GIDS—represents the next frontier. The goal is no longer just to make the neural network faster, but to ensure the entire pipeline, from raw data to prediction, scales linearly with the size of the graph.

Papers in this version

Showing 60 of 278 papers.

History & citationVersion 2 · 2026-10-04

Current · Permanent link

  1. Version 2 (latest) · 2026-10-04 · 284 papers
  2. Version 1 · 2026-10-04 · 278 papers
Cite this direction

Latest version (a living page):

@misc{lacuna_preprocessing_overhead_pre_processing_ov_0c796b14,
  title = {preprocessing overhead (pre-processing overhead)},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/preprocessing-overhead-pre-processing-overhead/rdr_0c796b1462294dd5920b0baa3bd0e9a9},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-19},
}

This exact version (version 1, frozen text):

@misc{lacuna_preprocessing_overhead_in_scalable_graph_0c796b14_v1,
  title = {Preprocessing Overhead in Scalable Graph Neural Networks},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/preprocessing-overhead-in-scalable-graph-neural-networks/rdr_0c796b1462294dd5920b0baa3bd0e9a9/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-19},
}