Understanding and Tackling Over-Dilution in Graph Neural Networks
Lists
Only you see this
Abstract
Message Passing Neural Networks (MPNNs) have become the predominant architecture for representation learning on graphs. While they hold promise, several inherent limitations have been identified, such as over-smoothing and over-squashing. Both theoretical frameworks and empirical investigations substantiate these limitations, facilitating advancements for informative representation. In this paper, we investigate the limitations of MPNNs from a novel perspective. We observe that even in a single layer, a node's own information can become considerably diluted, potentially leading to negative effects on performance. To delve into this phenomenon in-depth, we introduce the concept of *Over-dilution* and formulate it with two types of dilution factors: *intra-node dilution* and *inter-node dilution*. *Intra-node dilution* refers to the phenomenon where attributes lose their influence within each node, due to being combined with equal weight regardless of their practical importance. *Inter-node dilution* occurs when the node representations of neighbors are aggregated, leading to a diminished influence of the node itself on the final representation. We also introduce a transformer-based solution, which alleviates over-dilution by merging attribute representations based on attention scores between node-level and attribute-level representations. Our findings provide new insights and contribute to the development of informative representations.
BibTeX
@inproceedings{lee2025understanding,
author = {Junhyun Lee and Veronika Thost and Bumsoo Kim and Jaewoo Kang and Tengfei Ma},
title = {{Understanding and Tackling Over-Dilution in Graph Neural Networks}},
booktitle = {ACM SIGKDD Conference on Knowledge Discovery and Data Mining},
year = {2025},
}