Confidence-based Graph Convolutional Networks for Semi-Supervised Learning

Shikhar Vashishth, Prateek Yadav, Manik Bhandari, Partha P. Talukdar

Abstract

Predicting properties of nodes in a graph is an important problem with applications in a variety of domains. Graph-based Semi Supervised Learning (SSL) methods aim to address this problem by labeling a small subset of nodes and propagating this information across the graph. However, standard Graph Convolutional Networks (GCNs) often struggle with noisy neighbors and the phenomenon of over-smoothing in deeper architectures.

Introduction

In standard Graph Convolutional Networks (GCNs), neighborhood aggregation is typically isotropic, meaning the model treats all neighbors with a similar level of influence based primarily on the graph's topology. While effective in homogeneous graphs, this approach falters in real-world, heterogeneous networks where a node may be surrounded by neighbors of different classes or outliers. In such cases, a node can be easily "swayed" by the majority label of its neighbors, even if those neighbors are incorrectly classified or irrelevant.

To address this, researchers from the Indian Institute of Science introduced ConfGCN, a novel framework that incorporates label uncertainty into the aggregation process. By estimating the confidence of each neighbor's label, the model can intelligently weight their influence, effectively filtering out noise.

Methodology: Confidence-Aware Aggregation

ConfGCN introduces two primary innovations to the GCN pipeline to enable anisotropic neighborhood aggregation:

  1. Joint Estimation of Labels and Confidence: Rather than predicting a simple label vector, ConfGCN estimates a label distribution (represented by mean $\mu$) and a diagonal covariance matrix ($\Sigma$) for every node. The inverse of the variance serves as a proxy for the model's confidence in a specific label prediction.
  2. Influence Scoring: The model calculates an influence score ($r_{uv}$) for every edge based on the symmetric Mahalanobis distance between the label distributions of the connected nodes. Neighbors with high variance (low confidence) or those with highly divergent label distributions see their influence reduced during the aggregation step.
ConfGCN vs. Kipf-GCN: ConfGCN utilizes label confidences to weight neighbor influence, allowing it to correctly predict the target label even when surrounded by noisy or incorrect neighbors.

To train the network, the authors utilized a multi-part loss function that balances standard classification error with "smoothness" (ensuring confident, similar neighbors remain close in the feature space) and "consistency" (aligning estimated distributions with final model predictions).

Experimental Results

Performance and Robustness

The model was evaluated on standard citation benchmarks (Cora, Citeseer, Pubmed) and a more challenging, noisy version of Cora (Cora-ML). ConfGCN consistently outperformed state-of-the-art baselines, achieving 72.7% accuracy on Citeseer (compared to 69.4% for standard GCN) and 86.5% on Cora-ML.

One of the most significant findings was ConfGCN's resilience to "over-smoothing." In standard GCNs, increasing the number of layers often leads to indistinguishable node representations and a sharp drop in accuracy. ConfGCN maintains stable performance even as the architecture grows deeper.

Accuracy vs. Number of Layers: ConfGCN demonstrates significantly better resilience to over-smoothing than standard GCNs, maintaining higher accuracy as the number of layers increases.

Handling Neighborhood Heterogeneity

The authors also analyzed performance relative to "Neighbor Label Entropy," a measure of how mixed a node's neighborhood is. As entropy increases—indicating a messier, more heterogeneous neighborhood—ConfGCN’s advantage over models like Graph Attention Networks (GAT) becomes more pronounced. This suggests that confidence-based weighting is a more robust mechanism for noise suppression than standard attention mechanisms.

Conclusion

ConfGCN represents a significant step forward in making Graph Convolutional Networks more robust for real-world applications. By explicitly modeling uncertainty and using it to guide neighborhood aggregation, the framework achieves state-of-the-art results while mitigating common pitfalls like over-smoothing and sensitivity to label noise. Future work may explore the use of full covariance matrices to capture complex label correlations and the application of confidence-awareness to tasks beyond node classification, such as link prediction.

Summary updated

Abstract

Predicting properties of nodes in a graph is an important problem with applications in a variety of domains. Graph-based Semi Supervised Learning (SSL) methods aim to address this problem by labeli...
BibTeX
@inproceedings{vashishth2019confidence,
  author = {Shikhar Vashishth and Prateek Yadav and Manik Bhandari and Partha P. Talukdar},
  title = {{Confidence-based Graph Convolutional Networks for Semi-Supervised Learning}},
  booktitle = {International Conference on Artificial Intelligence and Statistics},
  year = {2019},
  url = {https://openreview.net/forum?id=UXRy4djsEuk},
}
Figure