Meta-Aggregator: Learning to Aggregate for 1-bit Graph Neural Networks

Yongcheng Jing, Yiding Yang, Xinchao Wang, Mingli Song, Dacheng Tao

Introduction

Graph Neural Networks (GNNs) have become a standard for processing non-Euclidean data, yet their deployment on resource-constrained edge devices remains challenging due to high memory and computational demands. Binarization—reducing 32-bit weights and features to 1-bit values—offers a promising path toward ultra-lightweight GNNs. By utilizing bitwise operations like XNOR and bit-counting, these models can achieve significant speedups and memory savings.

However, standard binarization often leads to a severe drop in accuracy. The authors identify a fundamental cause: topology indistinguishability. In a discrete 1-bit feature space, traditional aggregation functions (e.g., Mean or Max) lose the ability to distinguish between different local graph structures, effectively "blinding" the network to the graph's topology.

The Meta-Aggregator Framework

To overcome this limitation, the authors propose the Meta-Aggregator, a framework that replaces fixed aggregation functions with learnable modules. Instead of relying on a single, pre-defined operation, the network learns the optimal way to aggregate information based on the specific graph structure and layer context.

Architecture of the proposed Meta-Aggregator framework, featuring GNA and ANA

The framework introduces two primary components:

  1. Greedy Gumbel Neighborhood Aggregator (GNA): GNA acts as an adaptive selector. It employs a binarized Graph Auto-Encoder to analyze neighborhood features and selects the most effective aggregator from a predefined pool (e.g., Mean, Max, Sum, Std). To ensure the discrete selection process is differentiable, the Gumbel-Softmax trick is utilized during training.
  2. Adaptable Hybrid Neighborhood Aggregator (ANA): ANA provides a more flexible, continuous approach. It uses a mathematical formulation capable of smoothly approximating multiple aggregation behaviors simultaneously. This allows the model to create "hybrid" aggregators tailored to the specific needs of the data.

Key Insights: The Indistinguishability Trap

The core motivation for the Meta-Aggregator is the observation that 1-bit features significantly reduce the discriminative power of standard aggregators. As shown in the figure below, different graph topologies can result in identical aggregated values when features are binarized, a phenomenon much less prevalent in full-precision models.

The topology indistinguishability problem in 1-bit GNNs

By learning the aggregation function itself, the Meta-Aggregator recovers the structural information lost during binarization, allowing 1-bit GNNs to maintain high performance.

Experimental Results

The Meta-Aggregator was evaluated on diverse benchmarks, including molecular regression (ZINC), node classification (PPI), and 3D object recognition (ModelNet40).

  • ZINC: The ANA module reduced the Mean Absolute Error (MAE) of a 1-bit GCN from 0.669 to 0.607, closing the gap toward the full-precision baseline (0.407).
  • ModelNet40: The framework achieved a remarkable performance boost, raising the accuracy of a 1-bit GNN from 74.19% to 84.64%. This 10% improvement brings the model within reach of the 32-bit performance (92.42%) while remaining up to 20x smaller in memory footprint.
  • Efficiency: Crucially, these gains do not sacrifice the efficiency of 1-bit networks. The meta-logic is implemented using binarized components, ensuring that the overall architecture remains suitable for edge deployment.

Conclusion and Future Work

The Meta-Aggregator demonstrates that the bottleneck in 1-bit GNNs is not just precision loss, but the failure of fixed aggregation functions in discrete spaces. By making aggregation learnable, the framework significantly enhances the capacity of ultra-low-bitwidth models. Future research will explore extending these concepts to other architectures, such as lightweight Vision Transformers, to address similar information bottlenecks in compact neural networks.

Summary updated

Abstract

In this paper, we study a novel meta aggregation scheme towards binarizing graph neural networks (GNNs). We begin by developing a vanilla 1-bit GNN framework that binarizes both the GNN parameters and the graph features. Despite the lightweight architecture, we observed that this vanilla framework suffered from insufficient discriminative power in distinguishing graph topologies, leading to a dramatic drop in performance. This discovery motivates us to devise meta aggregators to improve the expressive power of vanilla binarized GNNs, of which the aggregation schemes can be adaptively changed in a learnable manner based on the binarized features. Towards this end, we propose two dedicated forms of meta neighborhood aggregators, an exclusive meta aggregator termed as Greedy Gumbel Neighborhood Aggregator (GNA), and a diffused meta aggregator termed as Adaptable Hybrid Neighborhood Aggregator (ANA). GNA learns to exclusively pick one single optimal aggregator from a pool of candidates, while ANA learns a hybrid aggregation behavior to simultaneously retain the benefits of several individual aggregators. Furthermore, the proposed meta aggregators may readily serve as a generic plugin module into existing full-precision GNNs. Experiments across various domains demonstrate that the proposed method yields results superior to the state of the art.
BibTeX
@inproceedings{jing2021meta,
  author = {Yongcheng Jing and Yiding Yang and Xinchao Wang and Mingli Song and Dacheng Tao},
  title = {{Meta-Aggregator: Learning to Aggregate for 1-bit Graph Neural Networks}},
  booktitle = {IEEE/CVF International Conference on Computer Vision},
  year = {2021},
  doi = {10.1109/iccv48922.2021.00525},
}
Figure