AttPool: Towards Hierarchical Feature Representation in Graph Convolutional Networks via Attention Mechanism

Jingjia Huang, Zhangheng Li, Nannan Li, Shan Liu, Ge LiICCV 2019 · DOI

Abstract

Graph Convolutional Networks (GCNs) have revolutionized node-level representation learning but often struggle with graph-level classification tasks due to their "flat" aggregation schemes. We present AttPool, an end-to-end trainable pooling module that introduces hierarchical structure to GCNs using an attention mechanism. AttPool adaptively selects a subset of "important" nodes to form coarser graph representations while preserving critical structural information through attention-based aggregation. Our hierarchical architecture, which incorporates intermediate supervision, achieves state-of-the-art results across several benchmarks, notably improving accuracy by an average of 6.91% over standard GCNs.

Introduction

In computer vision, the success of Convolutional Neural Networks (CNNs) is largely attributed to their ability to learn hierarchical features through successive pooling layers. In contrast, many GCNs aggregate information globally, making it difficult to distinguish between local patterns and global structures. While methods like DiffPool have been proposed, they are often computationally intensive or risk losing structural integrity. AttPool addresses these challenges by framing pooling as a learnable selection problem, enabling the model to focus on discriminative nodes while maintaining a manageable computational footprint.

Methods

The AttPool module consists of two primary stages: Att-Pick and Att-Aggregation. During Att-Pick, the model learns a projection vector to score node importance and selects the top-$k$ nodes. Att-Aggregation then ensures that features from discarded nodes are integrated into the selected nodes' representations via attention-weighted neighborhood aggregation.

The overall hierarchical architecture of AttGCN, featuring multiple AttPool layers and deep supervision via intermediate classifiers.

A key innovation is the Local Attention mechanism, which calculates scores based on a node's 1-hop neighborhood and degree. This prevents "parochial selection"—the tendency of global attention to cluster selections in a single dense region—ensuring a more representative downsampling of the graph. Furthermore, the architecture utilizes a hierarchical prediction head, where classifiers at each level contribute to a final ensemble decision. This "deep supervision" facilitates faster convergence and allows the model to leverage multi-scale features for the final classification.

Results

We evaluated AttPool on six standard benchmarks, including bio-informatics datasets (NCI1, PROTEINS) and social networks (COLLAB, REDDIT). AttPool consistently outperformed both flat GCNs and advanced pooling methods like Graph U-Net and DiffPool.

Comparison of AttPool (Global and Local variants) against GCN, Graph U-Net, and DiffPool across three major datasets.

As shown in the performance comparison, the Local Attention variant (AttPool-L) achieved the highest accuracy across all tested datasets, reaching 93.15% on REDDIT-BINARY. This represents a significant improvement over the 85.20% GCN baseline. Qualitative analysis using t-SNE visualizations confirmed that the hierarchical layers progressively refine node embeddings, leading to significantly better class separation in deeper layers compared to non-hierarchical models.

Conclusion

AttPool demonstrates that attention-based node selection is a powerful tool for building hierarchical GCNs. By intelligently downsampling graphs and employing deep supervision, AttPool captures multi-scale structural information essential for complex graph classification. Future research will focus on making the pooling ratio adaptive and incorporating edge-specific features into the attention mechanism to further enhance representation learning.

Summary updated

BibTeX
@inproceedings{huang2019attpool,
  author = {Jingjia Huang and Zhangheng Li and Nannan Li and Shan Liu and Ge Li},
  title = {{AttPool: Towards Hierarchical Feature Representation in Graph Convolutional Networks via Attention Mechanism}},
  booktitle = {IEEE/CVF International Conference on Computer Vision},
  year = {2019},
  doi = {10.1109/iccv.2019.00658},
}
Figure