Predicting the Silent Majority on Graphs: Knowledge Transferable Graph Neural Network

Wendong Bi, Bingbing Xu, Xiaoqian Sun, Easton Li Xu, Huawei Shen, Xueqi Cheng

TL;DR

Most Graph Neural Networks (GNNs) fail when applied to "VS-Graphs," where a vocal minority has rich data while the silent majority lacks features and labels. This paper introduces KTGNN, a framework that bridges this gap by adaptively transferring knowledge from vocal nodes to silent ones through domain-aware feature completion, message passing, and classifier adaptation.

What Problem Does This Paper Address?

In many real-world networks, there is a massive disparity in data availability. Consider a political social network: famous politicians (the "vocal minority") have detailed profiles and clear party affiliations. In contrast, ordinary citizens (the "silent majority") often have sparse profiles and unknown leanings. Similarly, in financial networks, listed companies publish exhaustive financial statements, while unlisted ones do not.

Standard GNNs struggle here for two reasons:

  1. Feature Incompleteness: Silent nodes are missing critical data dimensions.
  2. Distribution Shift: Vocal nodes and silent nodes are fundamentally different (e.g., listed companies have vastly different asset scales than unlisted ones).

Existing GNNs typically assume all nodes are Independent and Identically Distributed (I.I.D.), leading to poor performance when trying to predict the "silent majority" using data from the "vocal minority."

Examples of VS-Graphs in political and financial contexts.

Methods

The authors propose the Knowledge Transferable Graph Neural Network (KTGNN), which consists of three main modules designed to handle the domain gap:

  • Domain Adapted Feature Complementor (DAFC): Instead of simple zero-filling or mean-imputation, DAFC iteratively completes missing features for silent nodes. It uses a learnable mechanism to "translate" unobservable features from vocal neighbors to silent ones while accounting for the observable distribution shift between the two groups.
  • Domain Adapted Message Passing (DAMP): Standard message passing mixes features indiscriminately. DAMP introduces a distribution-shift variable to calibrate messages. When a vocal node sends information to a silent node (or vice versa), the message is projected into the target node's domain to eliminate noise caused by out-of-distribution features.
  • Domain Transferable Classifier (DTC): Rather than forcing the model to learn "domain-invariant" features (a common but sometimes restrictive transfer learning tactic), DTC transfers the parameters of the classifier. It uses KL-divergence to constrain a generated target classifier to sit in the "ideal" spot between a source-trained classifier (which underfits the target) and a target-trained classifier (which overfits due to label scarcity).
Architecture overview of KTGNN showing DAFC, DAMP, and DTC modules.

Datasets

The model was evaluated on two distinct real-world VS-Graphs:

  • Company Dataset: 10,641 Chinese companies (listed vs. unlisted). The task is financial risk assessment (risky vs. normal). Listed companies have 78 unobservable features (financial statements) that unlisted companies lack.
  • Twitter Dataset: A political social network of politicians and civilians. The task is predicting political tendency (Democrat vs. Republican). Politicians have 1,536 unobservable features (homepage descriptions) missing for civilians.

Key Insights and Ideas

  • Distribution Shift is Measurable: The authors use single-variable box plots and multi-variable t-SNE visualizations to prove that vocal and silent nodes occupy different areas of the feature space. Standard GNNs that ignore this shift inevitably fail.
  • Preserving vs. Erasing Differences: Unlike many domain adaptation methods that try to make different domains look identical, KTGNN succeeds by preserving domain differences during message passing and only "translating" the knowledge where necessary.
  • The "Ideal" Classifier: Figure 5 illustrates a key insight: a classifier trained only on the source (vocal) domain underperforms on the target, while one trained on the sparse target (silent) labels overfits. The DTC module successfully finds a middle ground that generalizes better.
t-SNE visualization showing the clear distribution shift between vocal and silent nodes.

Key Results

KTGNN consistently outperformed nine representative baselines (including GCN, GAT, and OOD-specific GNNs) across both datasets:

  • Financial Risk (Company): KTGNN achieved an AUC of 67.11%, a nearly 6% improvement over the best-performing baseline (OODGAT).
  • Political Ideology (Twitter): KTGNN reached an AUC of 95.08%, significantly higher than standard GNNs like GCN (86.88%).
  • Ablation Studies: Removing any single component (DAFC, DAMP, or DTC) led to a performance drop, confirming that feature completion, calibrated message passing, and classifier transfer are all essential for tackling VS-Graphs.

Limitations and Future Work

While KTGNN shows strong performance, some areas remain for exploration:

  • Connectivity Dependency: The feature completion (DAFC) relies on iterative expansion from vocal nodes. If a silent node is completely isolated from the vocal minority in the graph topology, the transfer mechanism may be less effective.
  • Binary Focus: The current work focuses primarily on binary classification tasks. Extending this to multi-class or regression tasks in VS-Graphs would be a natural next step.
  • Computational Complexity: Although the authors note the complexity is on par with mainstream GNNs, the iterative nature of DAFC adds some overhead compared to simple imputation methods.

Summary updated

Abstract

Graphs consisting of vocal nodes ("the vocal minority") and silent nodes ("the silent majority"), namely VS-Graph, are ubiquitous in the real world. The vocal nodes tend to have abundant features and labels. In contrast, silent nodes only have incomplete features and rare labels, e.g., the description and political tendency of politicians (vocal) are abundant while not for ordinary people (silent) on the twitter's social network. Predicting the silent majority remains a crucial yet challenging problem. However, most existing message-passing based GNNs assume that all nodes belong to the same domain, without considering the missing features and distribution-shift between domains, leading to poor ability to deal with VS-Graph. To combat the above challenges, we propose Knowledge Transferable Graph Neural Network (KT-GNN), which models distribution shifts during message passing and representation learning by transferring knowledge from vocal nodes to silent nodes. Specifically, we design the domain-adapted "feature completion and message passing mechanism" for node representation learning while preserving domain difference. And a knowledge transferable classifier based on KL-divergence is followed. Comprehensive experiments on real-world scenarios (i.e., company financial risk assessment and political elections) demonstrate the superior performance of our method. Our source code has been open sourced.
BibTeX
@misc{bi2023predicting,
  author = {Wendong Bi and Bingbing Xu and Xiaoqian Sun and Easton Li Xu and Huawei Shen and Xueqi Cheng},
  title = {{Predicting the Silent Majority on Graphs: Knowledge Transferable Graph Neural Network}},
  year = {2023},
  eprint = {2302.00873},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2302.00873},
}
Figure