Abstract
Graph Neural Networks (GNNs) have been widely applied in the semi-supervised node classification task, where a key point lies in how to sufficiently leverage the limited but valuable label information. Most of the classical GNNs solely use the known labels for computing the classification loss at the output. In recent years, several methods have been designed to additionally utilize the labels at the input. One part of the methods augment the node features via concatenating or adding them with the one-hot encodings of labels, while other methods optimize the graph structure by assuming neighboring nodes tend to have the same label. To bring into full play the rich information of labels, in this paper, we present a label-enhanced learning framework for GNNs, which first models each label as a virtual center for intra-class nodes and then jointly learns the representations of both nodes and labels. Our approach could not only smooth the representations of nodes belonging to the same class, but also explicitly encode the label semantics into the learning process of GNNs. Moreover, a training node selection technique is provided to eliminate the potential label leakage issue and guarantee the model generalization ability. Finally, an adaptive self-training strategy is proposed to iteratively enlarge the training set with more reliable pseudo labels and distinguish the importance of each pseudo-labeled node during the model training process. Experimental results on both real-world and synthetic datasets demonstrate our approach can not only consistently outperform the state-of-the-arts, but also effectively smooth the representations of intra-class nodes.
Introduction: Beyond Output-Only Labels
In semi-supervised node classification, the primary challenge is to effectively utilize a small set of labeled nodes to predict the categories of a much larger unlabeled set. Standard Graph Neural Networks (GNNs), such as GCN or GAT, typically treat labels as targets for the loss function at the very end of the training pipeline. This "output-only" approach often overlooks the rich semantic information and structural potential inherent in the labels themselves.
While some recent advancements attempt to "inject" labels at the input stage—for instance, by concatenating one-hot label encodings to node features—these methods frequently struggle with two issues: label semantics (failing to capture how different classes relate) and label leakage (where the model simply memorizes input labels rather than learning generalizable features).
The LEGNN Framework
To address these limitations, the authors propose LEGNN (Label-Enhanced Graph Neural Network). This framework reimagines labels not just as targets, but as active components of the graph structure. The core workflow involves transforming the original graph into a heterogeneous one where labels act as "virtual centers."
Core Innovations
- Heterogeneous Graph Construction: LEGNN introduces "Label Nodes" into the graph. If a training node belongs to a specific class, an edge is created between that node and the corresponding label node. This effectively "short-circuits" the distance between nodes of the same class, ensuring they are at most two hops away from each other regardless of their original connectivity.
- Heterogeneous Message Passing: The model employs distinct parameters for different interaction types (node-to-node vs. node-to-label). This allows the GNN to learn explicit label embeddings that capture semantic correlations between classes (e.g., recognizing that "Machine Learning" is closer to "Optimization" than to "Biology").
- Training Node Selection (TNS): To prevent the model from trivially memorizing input labels (leakage), LEGNN uses a sampling strategy. In each epoch, labeled nodes are split: one subset provides the ground truth for loss, while the other is used to build edges to label nodes. This forces the model to learn to predict labels without having a direct, "cheating" edge to the target.
- Adaptive Self-Training (AS-Train): Unlike standard self-training that uses fixed thresholds, AS-Train uses "Training Confidence" to determine when the model is mature enough to generate pseudo-labels and "Evaluating Confidence" to weight those labels during training, preventing error propagation.
Experimental Evaluation
LEGNN was evaluated on several large-scale benchmarks, including ogbn-arxiv, ogbn-mag, and oag-venue.
- Performance: LEGNN consistently outperformed vanilla GNN backbones and existing label-augmentation methods. For example, on the massive
ogbn-magdataset, LEGNN + AS-Train achieved 53.78% accuracy, a significant improvement over the standard GAT (~49%). - Smoothing Effect: Using the "Graph Difference" (GD) metric, the authors demonstrated that LEGNN produces significantly tighter clusters for intra-class nodes, proving its ability to smooth representations effectively.
- Robustness: In synthetic tests (
syn-arxiv), LEGNN maintained stable performance even in graphs with low homophily (where neighbors often have different labels), where traditional GNNs typically fail.
Conclusion and Future Work
LEGNN represents a significant step forward in semi-supervised learning on graphs by treating labels as structural entities. While the model introduces additional complexity—roughly a 1.23x increase in inference time and a 2.05x increase in parameters—the gains in accuracy and representation smoothing are substantial. Future research may explore more complex initial features for label nodes and extend this framework to tasks like link prediction or graph-level classification.
Summary updated