Abstract
Predicting properties of nodes in a graph is an important problem with applications in a variety of domains. Graph-based Semi-Supervised Learning (SSL) methods aim to address this problem by labeling a small subset of the nodes as seeds and then utilizing the graph structure to predict label scores for the rest of the nodes in the graph. Recently, Graph Convolutional Networks (GCNs) have achieved impressive performance on the graph-based SSL task. In addition to label scores, it is also desirable to have confidence scores associated with them. Unfortunately, confidence estimation in the context of GCN has not been previously explored. We fill this important gap in this paper and propose ConfGCN, which estimates labels scores along with their confidences jointly in GCN-based setting. ConfGCN uses these estimated confidences to determine the influence of one node on another during neighborhood aggregation, thereby acquiring anisotropic capabilities. Through extensive analysis and experiments on standard benchmarks, we find that ConfGCN is able to outperform state-of-the-art baselines.
Introduction
In graph-based semi-supervised learning, the goal is to predict labels for unlabeled nodes given a small set of labeled "seed" nodes. Standard Graph Convolutional Networks (GCNs), such as the Kipf-GCN, aggregate information from a node's neighbors based primarily on the graph structure. However, these models typically treat all neighbors with similar importance, ignoring the model's own certainty about those neighbors' labels.
In real-world scenarios, graphs often contain heterogeneous neighborhoods where a node may be connected to neighbors of different classes. A standard GCN might be "swayed by the majority" in such cases, leading to incorrect classifications. To address this, GCNs require anisotropic capabilities—the ability to vary the strength of influence based on the quality or confidence of the information provided by each neighbor.
Methods: Confidence-Aware Aggregation
The proposed ConfGCN framework moves beyond simple point estimates for labels by jointly estimating label scores and their associated confidences.
Joint Estimation and Influence Scoring
For every node, ConfGCN estimates both a label distribution ($\mu$) and a diagonal covariance matrix ($\Sigma$). The inverse of this covariance represents the model's confidence in its prediction. During neighborhood aggregation, instead of simple averaging, ConfGCN calculates an influence score ($r_{uv}$) between nodes. This score is derived from a symmetric Mahalanobis distance that incorporates the confidences of both the central node and its neighbor. If a neighbor has a high-variance (low-confidence) label, its influence on its neighbors is automatically subdued, preventing "noisy" information from corrupting node representations.
Multi-Term Loss Function
The model is trained using a composite loss function designed to balance classification accuracy with confidence estimation:
- Cross-entropy: Standard loss on labeled seed nodes.
- Smoothness: Encourages neighboring nodes to share similar label distributions.
- Label Consistency: Ensures predicted distributions for seeds remain close to ground truth.
- Covariance Regularization: Maintains finite and positive confidence estimates.
- Prediction Constraint: Aligns the estimated label distribution with the final model output.
Experimental Results
ConfGCN was evaluated on standard citation network benchmarks (Cora, Citeseer, Pubmed) and Cora-ML, a dataset characterized by a high label mismatch rate.
Performance and Robustness
ConfGCN consistently outperformed state-of-the-art baselines like Kipf-GCN and Graph Attention Networks (GAT). For instance, on the Citeseer dataset, ConfGCN achieved 72.7% accuracy compared to Kipf-GCN's 69.4%. On the heterogeneous Cora-ML dataset, it reached 86.5% accuracy, demonstrating superior robustness to neighborhood noise.
Robustness to Depth
A significant challenge for GCNs is the "over-smoothing" effect, where performance drops as more layers are added. As shown in the figure above, ConfGCN's performance degrades much more gracefully than Kipf-GCN as depth increases. The confidence mechanism effectively filters out noise that would otherwise accumulate through multiple rounds of aggregation, allowing the model to maintain signal integrity in deeper architectures.
Conclusion
ConfGCN introduces a principled way to incorporate label uncertainty into graph neural networks. By enabling anisotropic neighborhood aggregation through joint confidence estimation, the model achieves state-of-the-art results and exhibits enhanced robustness to both neighborhood heterogeneity and network depth. Future work includes optimizing computational complexity for massive graphs and extending the framework to directed or multi-relational graph structures.
Summary updated