Abstract
Various graph contrastive learning models have been proposed to improve the performance of learning tasks on graph datasets in recent years. While effective and prevalent, these models are usually carefully customized. In particular, although all recent researches create two contrastive views, they differ greatly in view augmentations, architectures, and objectives. It remains an open question how to build your graph contrastive learning model from scratch for particular graph learning tasks and datasets. In this work, we aim to fill this gap by studying how graph information is transformed and transferred during the contrastive learning process and proposing an information-aware graph contrastive learning framework called InfoGCL. The key point of this framework is to follow the Information Bottleneck principle to reduce the mutual information between contrastive parts while keeping task-relevant information intact at both the levels of the individual module and the entire framework so that the information loss during graph representation learning can be minimized. We show for the first time that all recent graph contrastive learning methods can be unified by our framework. We empirically validate our theoretical analysis on both node and graph classification benchmark datasets, and demonstrate that our algorithm significantly outperforms the state-of-the-arts.
Introduction
Contrastive learning has revolutionized graph representation learning by enabling models to learn robust features from unlabeled data. The standard approach involves creating two "views" of a graph through augmentations, encoding them, and maximizing the agreement between their resulting representations.
Despite its success, Graph Contrastive Learning (GCL) has largely remained a "trial-and-error" field. Researchers have employed a wide variety of augmentations (e.g., node dropping vs. edge perturbation), encoders (e.g., GCN vs. GIN), and contrastive objectives without a unified theory explaining why specific combinations succeed. This lack of theoretical grounding is particularly problematic for graph data, where structural information and feature distributions vary significantly across domains, such as social networks versus chemical molecules. InfoGCL addresses this gap by applying the Information Bottleneck (IB) principle to provide a systematic foundation for GCL design.
The InfoGCL Framework
InfoGCL decouples the GCL pipeline into three sequential modules, each optimized through the lens of Information Bottleneck theory. The IB principle posits that an optimal representation should be a "minimal sufficient statistic"—it must retain all information necessary to predict a target task ($y$) while discarding as much irrelevant input noise as possible.
1. View Augmentation
The framework suggests that optimal views should share the minimum amount of information required for the downstream task. By selecting augmentations that maximize task-relevant information while minimizing the mutual information between the two views, the model is forced to learn the most essential semantic features.
2. View Encoding
The encoder is designed to be "task-relevant and simple." It aims to keep the information shared by the two views intact while minimizing the total information extracted from the raw input. This prevents the model from over-fitting on noise present in the original graph structure.
3. Representation Contrasting
After encoding, the representations are contrasted. The authors evaluated multiple modes (e.g., global-global, local-global) and determined that the optimal mode is the one that retains the maximum task-relevant information after aggregation. Notably, the "local-global" mode—contrasting node-level representations with graph-level ones—generally extracts superior structural information.
Information Bottleneck Principles
The core of InfoGCL lies in managing the overlap of information between the views, the original graph, and the target task. This is visualized through Venn diagrams where the goal is to isolate the intersection of task-relevant information while shedding extraneous data.
This theoretical approach leads to the "Minimal Information Paradox": for contrastive learning to be most effective, the two views should be as different as possible, provided they still share the core semantic information required for the task.
Experimental Validation
InfoGCL was validated across a diverse set of benchmarks, including graph classification (MUTAG, PTC-MR, IMDB, NCI1, COLLAB) and node classification (Cora, Citeseer, Pubmed). These datasets cover a wide spectrum of graph densities and structural properties.
Key Results
- Performance: InfoGCL consistently outperformed or matched state-of-the-art (SOTA) unsupervised methods. In graph classification, it achieved up to a 5.2% relative improvement in accuracy over previous unsupervised SOTA, even surpassing supervised kernels on certain datasets.
- Unification: The paper demonstrates that existing methods like DGI, InfoGraph, and GRACE can be unified and explained as specific instances within the InfoGCL framework.
- Negative Samples: The study provides nuance on the necessity of negative samples. While they are not always required for social or chemical graphs, they are vital for sparse graphs (like citation networks) with high-dimensional features to prevent representation collapse.
Conclusion
InfoGCL moves graph contrastive learning away from heuristic-based design toward a principled, information-aware approach. By optimizing the flow of information through the augmentation, encoding, and contrasting stages, the framework achieves superior robustness and accuracy across domains. While reaching "exact optimality" remains a challenge due to real-world data noise, InfoGCL provides a practical and theoretically sound roadmap for building effective graph representation models.
Summary updated