At the heart of every Graph Neural Network (GNN) is the aggregation step: the process of collapsing a neighborhood of information into a single vector. While this seems like a simple implementation detail, the choice between summation and averaging (mean) represents a fundamental trade-off between a model's ability to count structural features and its ability to generalize across different graph scales.
The Case for Summation: Counting and Scale
In many domains, the absolute number of connections a node has is a primary signal. If you replace a sum with a mean, you lose the "count" information. For example, in Heterogeneous Graph Neural Networks for Malicious Account Detection, Chaochao Chen and Ziqi Liu found that for fraud detection, the total volume of shared hardware connections is a critical indicator of an attacker. A mean operator would wash out this volume, making a single shared device look the same as fifty.
Similarly, in combinatorial optimization, Maxime Gasse and Andrea Lodi demonstrated in Exact Combinatorial Optimization with Graph Convolutional Neural Networks that sum-convolutions are essential for identifying highly constrained variables. In a bipartite graph representing a linear program, the node degree tells the model how many constraints affect a variable—a scale-dependent feature that mean-pooling would destroy.
This principle extends to graph-level tasks. In Raising the Bar in Graph-level Anomaly Detection, Chen Qiu argues that additive pooling is superior for anomaly detection because the total size of a graph is often a key indicator of abnormality. By preserving the node count in the final embedding, the model can distinguish between small, typical subgraphs and large, suspicious ones.
The Case for Mean: Stability and Generalization
If summation is so expressive, why use mean aggregation at all? The answer lies in numerical stability and the "generalization gap."
When a model is trained on small graphs but tested on much larger or denser ones, sum-based aggregators can produce feature magnitudes far outside the range seen during training. Jaejun Lee and Joyce Jiyoung Whang show in PAC-Bayesian Generalization Bounds for Knowledge Graph Representation Learning that mean aggregation keeps the norm of the graph diffusion matrix low. This theoretical bound explains why mean-based models generalize better to unseen data: they are less sensitive to the specific density of the graph.
This stability is particularly vital for inductive reasoning, where the model must handle subgraphs of varying sizes. In Stability and Generalization Capability of Subgraph Reasoning Models for Inductive Knowledge Graph Completion, Minsung Hwang and colleagues prove that mean-based functions prevent extreme fluctuations in output scores as graph density increases, ensuring the model's logic remains consistent regardless of how many neighbors a node has.
Balancing the Two: Normalization Strategies
The research community has moved toward sophisticated normalization to get the "best of both worlds"—the expressivity of sums with the stability of means.
- Row Normalization: Mahalakshmi Sabanayagam proved in Representation Power of Graph Convolutions : Neural Tangent Kernel Analysis that row normalization is the most robust way to preserve class-specific information. By scaling edge weights to be independent of the node's degree distribution, the model can focus on community structures without being biased by how many connections a node possesses.
- Pre-normalization and Scaling: In scientific computing, where values can span multiple orders of magnitude, standard GNNs often fail. Haifeng Zou addressed this in AutoAMG($\theta$): An Auto-tuned AMG Method Based on Deep Learning for Strong Threshold by using specialized normalization within the readout function. This prevents numerical explosion when processing multiscale matrix data.
- Continuous Kernels: Newer architectures like CKGConv: General Graph Convolution with Continuous Kernels by Liheng Ma and Yingxue Zhang apply scaling terms during aggregation to handle irregular connectivity patterns, effectively bridging the gap between simple spatial sums and complex spectral methods.
Summary of Intuition
The choice of aggregator is a choice of what the model should "see." Use summation when the quantity of neighbors is a feature (fraud, constraints, size-based anomalies). Use mean or normalization when the identity of the neighbors matters more than their count, or when the model must remain stable across graphs of wildly different densities.