Contrastive Meta-Learning for Few-shot Node Classification

Song Wang, Zhen Tan, Huan Liu, Jundong Li

Introduction

Graph Neural Networks (GNNs) have demonstrated remarkable success in modeling relational data. However, their performance typically depends on the availability of large-scale labeled datasets. In many real-world applications—such as identifying users in emerging social communities or categorizing new research topics—labels are scarce. This "few-shot" node classification problem requires models to generalize to new classes given only a handful of examples.

While meta-learning (learning-to-learn) has emerged as a promising solution by training models on a series of episodic tasks, existing methods often struggle with two critical aspects: intra-class generalizability (ensuring nodes of the same class are close in embedding space) and inter-class generalizability (ensuring distinct classes are well-separated, even when they are structurally similar). This paper introduces COSMIC (COntrastive Similarity-sensitive MIx-up for node Classification), a framework designed to address these gaps through contrastive optimization and structural data augmentation.

The COSMIC Framework

COSMIC improves few-shot performance by refining how the model perceives class boundaries and node relationships. The framework is built upon three technical pillars:

1. Contrastive Two-Step Optimization

Traditional meta-learning often relies solely on cross-entropy loss to classify query nodes. COSMIC introduces a two-step optimization process within each training episode:

  • Step One: A contrastive loss is applied to the support set. This forces the encoder to align embeddings of nodes within the same class while pushing nodes from different classes apart. This ensures a well-structured embedding space before classification begins.
  • Step Two: A standard cross-entropy loss is applied to the query set to fine-tune the model’s classification accuracy based on the refined embeddings.

2. Similarity-Sensitive Subgraph Mix-up

To improve inter-class generalizability, COSMIC employs a novel data augmentation strategy. Instead of training on "easy" class boundaries, the model generates synthetic "hard" classes. It extracts subgraphs using Personalized PageRank (PPR) and calculates their structural similarity using the Bhattacharyya distance. By mixing subgraphs that are structurally similar but belong to different classes, COSMIC forces the GNN to learn more nuanced features, making the model more robust to subtle differences between unseen test classes.

3. Subgraph-Centric Encoding

Rather than treating nodes as isolated feature vectors, COSMIC represents each node through its local neighborhood (subgraph). This approach ensures that the structural context—the "where" and "how" a node connects to the graph—is inherently baked into the representation, which is vital for distinguishing classes in complex networks.

Experimental Evaluation

The authors evaluated COSMIC against several state-of-the-art baselines, including G-Meta, TENT, and Meta-GNN, across four diverse benchmarks:

  • CoraFull & DBLP: Citation networks with a high number of classes (70 and 137, respectively).
  • ogbn-arxiv: A massive citation network of Computer Science papers (over 169,000 nodes).
  • Coauthor-CS: A co-authorship graph based on the Microsoft Academic Graph.

Key Results

COSMIC consistently outperformed existing methods across all datasets and shot-counts. Notable highlights include:

  • Significant Accuracy Gains: In the CoraFull 5-way 1-shot task, COSMIC achieved an accuracy of 74.93%, a substantial improvement over the previous state-of-the-art (G-Meta) which achieved approximately 60.44%.
  • Resilience to Task Complexity: As the number of classes (the "way") increased, COSMIC showed a much smaller performance degradation compared to baselines, suggesting superior inter-class discrimination.
  • Computational Efficiency: Despite the added contrastive step, the use of efficient subgraph sampling (PPR) allowed COSMIC to maintain training times comparable to or faster than several existing models on large-scale graphs like ogbn-arxiv.

Discussion and Conclusion

The success of COSMIC highlights a fundamental principle in graph representation learning: Structure is Identity. By focusing on subgraph-centric encoding and creating "hard" training scenarios through similarity-sensitive mix-up, the model develops a finer resolution for class boundaries.

However, some limitations remain. COSMIC still requires a set of labeled "base" classes for the initial meta-training phase, making it less suitable for complete "cold-start" scenarios where no labeled data exists anywhere in the graph. Additionally, the model is sensitive to the subgraph size ($K_s$); subgraphs that are too small lack context, while those that are too large may introduce noise. Future research may explore zero-shot extensions or more adaptive subgraph sampling techniques to further enhance the framework's flexibility.

Summary updated

BibTeX
@inproceedings{wang2023contrastive,
  author = {Song Wang and Zhen Tan and Huan Liu and Jundong Li},
  title = {{Contrastive Meta-Learning for Few-shot Node Classification}},
  booktitle = {ACM SIGKDD Conference on Knowledge Discovery and Data Mining},
  year = {2023},
  doi = {10.1145/3580305.3599288},
}
Figure