DeeperGCN: All You Need to Train Deeper GCNs
Lists
Only you see this
Abstract
Graph Neural Networks (GNNs) have been drawing significant attention to the power of representation learning on graphs. Recent works developed frameworks to train very deep GNNs. Such works show impressive results in tasks like point cloud learning and protein interaction prediction. In this work, we study the performance of such deep models in large-scale graph datasets from the Open Graph Benchmark (OGB). In particular, we look at the effect of adequately choosing an aggregation function and its effect on final performance. Common choices of aggregation are mean, max, and sum. It has been shown that GNNs are sensitive to such aggregations when applied to different datasets. We systematically study this point on large-scale graphs and propose to alleviate it by introducing a novel Generalized Aggregation Function. Proposed aggregation functions extend beyond the commonly used ones. The generalized aggregation functions are fully differentiable, and thus their parameters can be learned in an end-to-end fashion. We show that deep residual GNNs equipped with generalized aggregation functions achieve state-of-the-art results in several benchmarks from OGB across tasks and domains.
BibTeX
@misc{li2020deepergcn,
author = {Guohao Li and Chenxin Xiong and Guocheng Qian and Ali Thabet and Bernard Ghanem},
title = {{DeeperGCN: All You Need to Train Deeper GCNs}},
year = {2020},
eprint = {2006.07739},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2006.07739},
}| DeepGCNs: Making GCNs Go as Deep as CNNs | IEEE TPAMI · 2023 |
| DEEPER-GXX: DEEPENING ARBITRARY GNNS | arXiv · 2021 |
| Simple and Deep Graph Convolutional Networks | ICML · 2020 |
| Revisiting "Over-smoothing" in Deep GCNs | arXiv · 2020 |
| Training Graph Neural Networks with 1000 Layers | ICML · 2021 |
| Old can be Gold: Better Gradient Flow can Make Vanilla-GCNs Great Again | arXiv · 2022 |