Introduction
Graph representation learning aims to map complex nodes into low-dimensional embeddings that preserve essential structural and attribute information. While supervised methods excel when labels are abundant, they often overfit on small datasets. Conversely, self-supervised learning (SSL) captures broad patterns but remains "task-blind," potentially emphasizing graph signals irrelevant to the final downstream objective. This paper introduces Task-Aware Contrastive Learning (XTCL), a framework designed to bridge this relevance gap by adaptively selecting positive pairs based on task-specific importance.
The Relevance Gap in Graph SSL
A fundamental challenge in graph machine learning is that different tasks rely on different graph signals. For instance, node classification in a citation network might depend heavily on local topology, whereas product recommendation might prioritize node attributes. Standard SSL treats all signals uniformly, leading to suboptimal representations that may focus on "noise" relative to the target task.
Methodology: Task-Aware Contrastive Loss (XTCL)
The core of the proposed framework is the XGBoost Sampler (XGSampler). Unlike traditional contrastive learning which uses fixed heuristics for positive sampling, XTCL learns to identify high-quality positive examples by evaluating various "semantic relations" such as Jaccard similarity, PageRank, and attribute distributions.
XGSampler and Boosting
The XGSampler utilizes a small amount of labeled data to train an ensemble of relation regression stumps. By maximizing the mutual information between the learned representations and the downstream task labels, the sampler assigns global importance weights to different graph signals. This adaptive sampling ensures that the contrastive loss pulls together nodes that are truly relevant to the task, even for unlabeled nodes in a semi-supervised setting.
Empirical Evaluation
The authors evaluated XTCL on five benchmark datasets, including citation networks (Cora, CiteSeer, PubMed) and product networks (Amazon Photo, Amazon Computers).
Performance and Data Efficiency
XTCL consistently outperformed state-of-the-art supervised and self-supervised models. Notably, the method demonstrates superior data efficiency, maintaining a significant performance lead even when training labels are scarce. On the Cora dataset, XTCL achieved a top-tier accuracy of 85.19%, outperforming popular SSL methods like DGI and SUGRL. In link prediction tasks, XTCL reached an AUC of 97.13% on CiteSeer, a marked improvement over the baseline GCN.
Robustness and Interpretability
A key advantage of XTCL is its interpretability. By examining the weights assigned by XGSampler, researchers can identify which graph signals drive performance for a specific task. Furthermore, semi-synthetic experiments demonstrated that XTCL is more robust to noise than traditional GCNs, maintaining stable performance even when label-signal correlations are perturbed.
Conclusion and Limitations
XTCL effectively addresses the task-blindness of traditional graph SSL. While the framework introduces some computational overhead—with a worst-case complexity of $O(|V|^2)$ for dense relations—the authors provide GPU-accelerated implementations to mitigate this. Future work includes extending the framework to heterogeneous graphs and developing purely unsupervised task-awareness mechanisms.
Summary updated