Abstract
Graph Convolutional Networks (GCNs) have revolutionized node-level representation learning but often struggle with graph-level classification tasks due to their "flat" aggregation schemes. We present AttPool, an end-to-end trainable pooling module that introduces hierarchical structure to GCNs using an attention mechanism. AttPool adaptively selects a subset of "important" nodes to form coarser graph representations while preserving critical structural information through attention-based aggregation. Our hierarchical architecture, which incorporates intermediate supervision, achieves state-of-the-art results across several benchmarks, notably improving accuracy by an average of 6.91% over standard GCNs.
Introduction
In computer vision, the success of Convolutional Neural Networks (CNNs) is largely attributed to their ability to learn hierarchical features through successive pooling layers. In contrast, many GCNs aggregate information globally, making it difficult to distinguish between local patterns and global structures. While methods like DiffPool have been proposed, they are often computationally intensive or risk losing structural integrity. AttPool addresses these challenges by framing pooling as a learnable selection problem, enabling the model to focus on discriminative nodes while maintaining a manageable computational footprint.
Methods
The AttPool module consists of two primary stages: Att-Pick and Att-Aggregation. During Att-Pick, the model learns a projection vector to score node importance and selects the top-$k$ nodes. Att-Aggregation then ensures that features from discarded nodes are integrated into the selected nodes' representations via attention-weighted neighborhood aggregation.
A key innovation is the Local Attention mechanism, which calculates scores based on a node's 1-hop neighborhood and degree. This prevents "parochial selection"—the tendency of global attention to cluster selections in a single dense region—ensuring a more representative downsampling of the graph. Furthermore, the architecture utilizes a hierarchical prediction head, where classifiers at each level contribute to a final ensemble decision. This "deep supervision" facilitates faster convergence and allows the model to leverage multi-scale features for the final classification.
Results
We evaluated AttPool on six standard benchmarks, including bio-informatics datasets (NCI1, PROTEINS) and social networks (COLLAB, REDDIT). AttPool consistently outperformed both flat GCNs and advanced pooling methods like Graph U-Net and DiffPool.
As shown in the performance comparison, the Local Attention variant (AttPool-L) achieved the highest accuracy across all tested datasets, reaching 93.15% on REDDIT-BINARY. This represents a significant improvement over the 85.20% GCN baseline. Qualitative analysis using t-SNE visualizations confirmed that the hierarchical layers progressively refine node embeddings, leading to significantly better class separation in deeper layers compared to non-hierarchical models.
Conclusion
AttPool demonstrates that attention-based node selection is a powerful tool for building hierarchical GCNs. By intelligently downsampling graphs and employing deep supervision, AttPool captures multi-scale structural information essential for complex graph classification. Future research will focus on making the pooling ratio adaptive and incorporating edge-specific features into the attention mechanism to further enhance representation learning.
Summary updated