CoCoS: Enhancing Semi-supervised Learning on Graphs with Unlabeled Data via Contrastive Context Sharing

Siyue Xie, Da Sun Handason Tam, Wing Cheong LauAAAI 2022 · DOI

Abstract

Semi-supervised learning on graphs often suffers from extreme label sparsity, where standard Graph Neural Networks (GNNs) fail to utilize the vast majority of unlabeled nodes. We present CoCoS (Contrastive Context Sharing), a plug-and-play framework designed to bridge this gap. CoCoS enables nodes with similar semantic roles to share neighborhood information, regardless of their distance in the graph. By combining intra-class context sharing with contrastive learning, CoCoS produces more robust and discriminative representations, significantly outperforming existing GNN backbones and contrastive learning baselines.

Introduction

In many real-world graph applications, labeled data is scarce, often comprising less than 1% of the total nodes. Standard GNNs rely on local neighborhood aggregation, which limits their receptive field. Increasing the number of layers to capture long-range dependencies often leads to the over-smoothing problem, where node features become indistinguishable as they are repeatedly averaged across the network. Consequently, a significant portion of the graph remains underutilized. CoCoS addresses this by allowing semantically similar nodes—even those far apart—to "share" their contexts, effectively expanding the learning signal beyond immediate neighborhoods.

Methodology

The CoCoS framework operates through a four-step iterative process designed to leverage unlabeled data via semantic augmentation:

  1. Label Estimation: An initial GNN is pretrained on the available sparse labels to generate pseudo-labels for all unlabeled nodes.
  2. Intra-class Context Sharing: A "context-consistent graph" is constructed. Features are randomly shuffled among nodes predicted to belong to the same class. This assumes that nodes of the same class share compatible contexts, creating a semantically-preserving augmentation.
  3. Contrastive Pair Construction: The framework identifies positive pairs (the same node in the original vs. augmented graph) and negative pairs (nodes with different predicted labels).
  4. Joint Optimization: The model is trained using a dual objective: a standard classification loss on labeled nodes and a contrastive loss that maximizes similarity for positive pairs while minimizing it for negative pairs.
The four-step CoCoS framework for contrastive context sharing.

The estimated labels are updated progressively during training, allowing the model to refine its context-sharing logic as its predictive accuracy improves.

Experimental Setup

The effectiveness of CoCoS was evaluated on several benchmark citation networks:

  • Small-to-Medium Scale: Cora, Citeseer, and Pubmed.
  • Large Scale: Ogbn-arxiv, providing a test of scalability and performance in realistic scenarios.

Key Insights

  • Contextual Compatibility: The core premise is that a node's role is defined by its context (neighborhood topology and features). By shuffling features within classes, CoCoS allows shallow GNNs to learn from "remote" but semantically related nodes without the risks of over-smoothing.
  • Semantic Augmentation: Unlike traditional graph augmentations (like edge dropping), intra-class shuffling serves as a targeted, semantically-aware perturbation that forces the model to learn generalizable patterns rather than memorizing specific neighborhoods.
  • Architectural Flexibility: CoCoS is a model-agnostic wrapper that can be integrated with various backbones like GCN, GAT, or GraphSAGE with minimal computational or parameter overhead.

Key Results

Empirical results demonstrate that CoCoS consistently enhances the performance of underlying GNN architectures. Notably, it improved standard GCN accuracy by over 2% on the Cora, Citeseer, and Pubmed datasets.

Performance comparison of different CoCoS variants across standard benchmarks.

The framework proved particularly robust in extreme label-sparsity settings and outperformed dedicated Graph Contrastive Learning (GCL) methods such as DGI and MVGRL. Furthermore, the increase in learnable parameters remains marginal, typically under 10%, making it practical for real-world deployment.

Conclusion and Future Work

CoCoS provides a simple yet effective mechanism for utilizing unlabeled data in graph semi-supervised learning. While currently dependent on the quality of initial label estimations, future iterations could incorporate confidence-based weighting to mitigate noise from incorrect pseudo-labels. Additionally, extending this context-sharing principle to fully unsupervised settings remains a promising direction for future research.

Summary updated

BibTeX
@inproceedings{xie2022cocos,
  author = {Siyue Xie and Da Sun Handason Tam and Wing Cheong Lau},
  title = {{CoCoS: Enhancing Semi-supervised Learning on Graphs with Unlabeled Data via Contrastive Context Sharing}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2022},
  doi = {10.1609/aaai.v36i4.20347},
}
Figure