Introduction
Graph Neural Networks (GNNs) have become the gold standard for Collaborative Filtering (CF) due to their ability to capture "high-order" relationships—such as a friend of a friend's preferences. However, deploying these models in production environments reveals two major "Achilles' heels": over-smoothing (where user embeddings become indistinguishable) and scalability bottlenecks (recursive message passing is computationally expensive). SimRec addresses these challenges by proposing a "graph-less" framework that distills GNN-level relational power into a lightweight Multilayer Perceptron (MLP) student.
The Challenge: GNN Bottlenecks
While GNNs excel at modeling complex graphs, they are often "noisy," indiscriminately propagating signals from accidental clicks or popularity biases. As more layers are added to capture deeper relationships, the embeddings tend to converge, losing the unique preference signals necessary for personalization. Furthermore, at the scale of millions of users and items, the memory and time required for GNN inference become prohibitive for real-time applications.
Methodology: SimRec Architecture
SimRec utilizes a teacher-student architecture to "compress" graph knowledge into a non-graph model, allowing for efficient inference without the graph scaffold.
- Teacher-Student Paradigm: A lightweight Graph Convolutional Network (GCN) teacher learns from the full interaction graph, while a simple, fast MLP student is trained to mimic its behavior.
- Bi-Level Knowledge Distillation: The student learns through both prediction-level distillation (mimicking ranking scores) and embedding-level distillation (aligning hidden representations using InfoNCE contrastive loss).
- Adaptive Contrastive Regularization: This is the framework's core novelty. SimRec uses a gradient-based recalibration module to detect when the teacher provides over-smoothed or noisy signals. If the distillation task contradicts the primary recommendation task, the model adaptively adjusts weights to ensure user embeddings remain distinct and diverse.
Key Insights: Embedding Uniformity
A striking insight from the research is that the graph is a training scaffold, not a permanent requirement. By visualizing the latent space, the authors demonstrate that SimRec produces a more uniform spread of users compared to traditional GNNs.
This uniformity indicates that SimRec effectively avoids over-smoothing, maintaining the unique preference signals of individual users rather than clumping them together in the latent space.
Key Results: Accuracy and Scalability
SimRec consistently outperformed state-of-the-art GNN models, including LightGCN, SGL, and HCCF, across standard benchmarks like Gowalla, Yelp, and Amazon.
- Superior Accuracy: On the Amazon dataset, SimRec improved Recall@20 by over 10% compared to the best baseline.
- Massive Speedup: On the large-scale Tmall dataset (40 million records), SimRec was approximately 2x faster in inference time than competitive GNN models (785s vs 1331s for SGL) while maintaining higher accuracy.
- Robustness: Using the Mean Average Distance (MAD) metric, the authors confirmed that SimRec maintains significantly higher embedding diversity than traditional GNNs, successfully mitigating over-smoothing.
Conclusion
SimRec proves that the high-order relational insights of GNNs can be successfully distilled into lightweight models that do not require a graph during inference. By marrying knowledge distillation with adaptive contrastive learning, SimRec offers a scalable, robust solution for modern recommender systems. Future work will explore extending this graph-less approach to dynamic and temporal recommendation settings.
Summary updated