Dense Connectivity in Deep Graph Convolutional Networks

This direction is merged into another direction: graph over-smoothing (over-smoothing). Its last version stays readable and citable here.

Version 1 of 119 papers22 conceptsfrozen 2026-10-04

Graph Convolutional Networks (GCNs) traditionally suffer from a "shallow" limitation. While vision models like ResNet or DenseNet scale to hundreds of layers, GCNs historically struggled beyond three or four. As layers are added, node representations tend to converge to a single point—a phenomenon known as over-smoothing—making nodes indistinguishable and the network useless.

To overcome this, researchers have adapted dense connectivity and multi-layer aggregation to the graph domain. By linking every layer to every subsequent layer, these architectures ensure that local, fine-grained features from early layers are preserved even as the network extracts global, high-level structural information.

The Problem of Over-smoothing

In a standard GCN, each layer performs a message-passing operation where a node aggregates features from its immediate neighbors. After $k$ layers, a node’s representation is influenced by its $k$-hop neighborhood.

The intuition behind over-smoothing is that repeated averaging acts like a low-pass filter. As Kyuyong Shin and colleagues explain in Graphs, Entities, and Step Mixture for Enriching Graph Representation, node representations eventually converge to a mean value, losing the distinctiveness required for classification. This "information wash-out" is particularly problematic for graph-level tasks where structural nuances are critical.

Preserving Identity through Dense Connections

The core solution is to change how information flows through the network. Instead of a simple linear chain, dense connectivity creates a "highway" for features.

In DeepGCNs: Can GCNs Go As Deep As CNNs?, Guohao Li and Ali Thabet demonstrated that adapting dense connections—where the output of every preceding layer is concatenated as input to all subsequent layers—allows GCNs to reach depths of 56 layers. This architecture promotes feature reuse. Because the original node features and early-layer embeddings remain accessible at the final layer, the model can balance local details with broader contextual information.

This mechanism works for several reasons:

  1. Gradient Propagation: As noted by Asiri Wijesinghe in the DFNets framework, dense connectivity ensures robust gradient flow, preventing the vanishing gradient problem that plagues deep architectures.
  2. Multi-scale Connectivity: Shen Wang and Zhengzhang Chen argue in DefNet that this multi-scale approach also provides a defense against adversarial noise. If noise is introduced at a specific layer, the direct connections from other layers help maintain the integrity of the signal.

Multi-layer Aggregation Strategies

While simple concatenation is common, researchers have explored more sophisticated ways to fuse these multi-depth representations.

In AMA-GCN, Fuzhen Zhuang and Hao Chen utilize an adaptive strategy combining max-pooling and concatenation. This allows the model to selectively aggregate node representations from different depths, ensuring that the final embedding captures the most relevant structural features for specific tasks like disease prediction.

Similarly, the GCGCN model developed by Huiwei Zhou and Yibin Xu uses hierarchical graph blocks. By stacking these blocks and using dense feature concatenation, they preserve multi-level information—moving from local co-occurrence to global attention—without the risk of over-smoothing.

Beyond Fixed Topologies

Recent work has even begun to treat the network's connectivity itself as a learnable or generative property. Yang Yang and Zhengmin Kong introduced Kronecker Generative Networks, which use algebraic frameworks to generate complex topologies. These networks use weighted sums and biased residual connections to manage information flow across multiple paths, ensuring that even in highly complex, deep structures, feature distinctness is maintained.

By shifting from simple sequential processing to dense, multi-path architectures, these researchers have transformed GCNs from shallow aggregators into deep, robust models capable of capturing the full complexity of graph-structured data.

Papers in this version

History & citationVersion 1 · 2026-10-04

Merged into another direction · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_dense_connectivity_in_deep_graph_convolu_045f6462,
  title = {Dense Connectivity in Deep Graph Convolutional Networks},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/dense-connectivity-in-deep-graph-convolutional-networks/rdr_045f64628a334190b72e88688162c2dd},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-22},
}

This exact version (version 1, frozen text):

@misc{lacuna_dense_connectivity_in_deep_graph_convolu_045f6462_v1,
  title = {Dense Connectivity in Deep Graph Convolutional Networks},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/dense-connectivity-in-deep-graph-convolutional-networks/rdr_045f64628a334190b72e88688162c2dd/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-22},
}