Abstract
Semi-supervised inductive learning on graphs remains a significant challenge due to the scarcity of labeled data and the need for generalization to unseen graph structures. This paper introduces SLA-VGAE, a Variational Graph Auto-Encoder framework that prioritizes the reconstruction of node labels and features over graph topology. By integrating a node-masked pseudo-labeling strategy called SLAM, the model effectively leverages unlabeled data to improve classification accuracy. Experimental results on large-scale datasets like Flickr and Reddit demonstrate that SLA-VGAE outperforms existing GNN and VGAE baselines, particularly in low-label scenarios.
Introduction
Most Graph Neural Networks (GNNs) are designed for transductive learning, where the model predicts labels for unlabeled nodes within a fixed, known graph. However, real-world applications often require inductive learning—the ability to generalize to entirely new nodes or graph structures. Standard GNNs are typically "label-hungry," making them difficult to deploy when annotation is expensive. While Variational Graph Auto-Encoders (VGAEs) offer better generalization through generative regularization, they traditionally focus on reconstructing graph edges, which can lead to "proximity overfitting" and suboptimal classification performance.
Methodology
To address these limitations, the authors propose two primary innovations: the SLA-VGAE architecture and the SLAM augmentation method.
SLA-VGAE Architecture
Unlike traditional VGAEs, SLA-VGAE shifts the reconstruction objective. The encoder uses a standard GCN to produce latent representations ($Z$). The decoder then utilizes two distinct heads to reconstruct node labels ($\hat{Y}$) and node features ($\hat{X}$). By forcing the model to reconstruct labels, the latent space is optimized specifically for classification tasks rather than mere topological memorization.
Self-Label Augmentation Method (SLAM)
To mitigate label scarcity, SLAM employs a two-stage training process. After an initial warm-up phase, the model generates pseudo-labels for unlabeled nodes using node-wise masking. By randomly hiding nodes and predicting their labels based on neighbors over multiple iterations, the model learns robust structural patterns. Only predictions exceeding a specific confidence threshold ($\theta$) are retained as augmented labels.
Experimental Results
The model was evaluated on the Flickr and Reddit datasets, focusing on "low-label" regimes (1%, 10%, and 100% labeling rates).
SLA-VGAE consistently demonstrated superior performance and robustness. For instance, on the Flickr dataset with only 1% of labels, SLA-VGAE achieved an accuracy of 0.475, significantly outperforming standard GCNs (0.428) and other generative models like MaskGAE (0.427). On the Reddit dataset, the model showed remarkable stability; while competing models like GAMLP saw performance drops of approximately 35% when moving from 100% to 1% labeling, SLA-VGAE's accuracy remained highly resilient.
Discussion and Conclusion
The success of SLA-VGAE highlights three key insights:
- Label Reconstruction: Prioritizing label and feature reconstruction over edge reconstruction creates a latent space better suited for semi-supervised classification.
- Robust Masking: Node-wise masking prevents the model from memorizing labels, forcing it to learn the underlying graph logic.
- Generative Regularization: The KL divergence term acts as a data-agnostic regularizer, preventing weight collapse when labels are scarce.
While the method introduces some computational overhead due to the iterative pseudo-labeling process, its ability to maintain high accuracy with minimal supervision makes it a powerful tool for large-scale, real-world inductive graph learning.
Summary updated