Introduction
In modern social networks, user connectivity typically follows a power-law distribution. While a small group of "head users" possesses hundreds of connections, the vast majority of "tail users" have very few. Current Graph Neural Network (GNN) based recommendation systems suffer from a significant "degree-related bias," performing poorly for these underserved tail users. This performance gap stems from two primary challenges: Label Sparsity, where few observed friendships exist to guide training, and Neighborhood Sparsity, where the lack of local structure makes it difficult for models to generalize preferences from well-connected users to sparse ones.
To address these issues, we present Tail-STEAK (Tail user oriented Self-Training EnhAnced Knowledge distillation), a framework designed to bridge the gap between head and tail user performance through iterative self-training and a novel knowledge distillation approach.
Methodology
Tail-STEAK operates through a two-stage process that combines self-training with self-knowledge distillation tasks to synthesize robust user representations.
Two-Stage Self-Training
To mitigate label sparsity, the framework first trains a base model exclusively on head users and their established connections. In the second stage, this model generates "pseudo-links" for tail users—predicting potential friendships based on learned patterns. These high-confidence predictions are then incorporated as additional training labels, effectively enriching the sparse data available for tail users.
Self-Knowledge Distillation and MI Maximization
To bridge the preference distribution gap, Tail-STEAK employs data augmentation to create synthetic views of users:
- Head-to-Tail: Head users are transformed into synthetic tail users by aggressively dropping links and disturbing ID embeddings.
- Tail-to-Head: Tail users are transformed into synthetic head users by imputing their neighborhoods with predicted pseudo-links.
The framework utilizes Mutual Information (MI) Maximization to ensure consistency between these real and synthetic views. By forcing the model to align the distributions of a user's "head-like" and "tail-like" representations, it learns features that are invariant to the user's degree, leading to more robust recommendations.
Experimental Evaluation
The framework was evaluated on two real-world datasets: Deezer (28k nodes) and Last.FM (136k nodes). Both datasets exhibit strong power-law distributions. Tail-STEAK was implemented as a plug-and-play enhancement for two state-of-the-art GNN architectures: LightGCN and Simple-HGN (SHGN).
Results and Analysis
Experimental results demonstrate that Tail-STEAK significantly outperforms existing de-biasing and contrastive learning baselines, particularly for tail users.
Key findings include:
- Significant Tail Improvement: On the Last.FM dataset using LightGCN, Tail-STEAK improved the Mean Reciprocal Ranking (MRR) for tail users by 58.37% compared to the base model.
- Preservation of Head Performance: Unlike many methods that improve tail performance at the expense of head users, Tail-STEAK maintains or even slightly improves performance for well-connected users.
- Robustness of MI Maximization: The use of MI maximization proved superior to standard reconstruction losses, as it focuses on high-level distributional consistency rather than noisy feature-level details.
Conclusion
Tail-STEAK effectively addresses degree-related bias in social recommendations by synthesizing information across the head-tail divide. While the two-stage process adds some computational complexity and requires careful tuning of the degree threshold ($T$), its model-agnostic nature makes it a powerful tool for improving fairness and accuracy in sparse network environments. Future work may explore extending this approach to other link-prediction domains, such as knowledge graph completion.
Summary updated