Graph-less Collaborative Filtering

Lianghao Xia, Chao Huang, Jiao Shi, Yong Xu

Introduction

Graph Neural Networks (GNNs) have become the gold standard for Collaborative Filtering (CF) due to their ability to capture "high-order" relationships—such as a friend of a friend's preferences. However, deploying these models in production environments reveals two major "Achilles' heels": over-smoothing (where user embeddings become indistinguishable) and scalability bottlenecks (recursive message passing is computationally expensive). SimRec addresses these challenges by proposing a "graph-less" framework that distills GNN-level relational power into a lightweight Multilayer Perceptron (MLP) student.

The Challenge: GNN Bottlenecks

While GNNs excel at modeling complex graphs, they are often "noisy," indiscriminately propagating signals from accidental clicks or popularity biases. As more layers are added to capture deeper relationships, the embeddings tend to converge, losing the unique preference signals necessary for personalization. Furthermore, at the scale of millions of users and items, the memory and time required for GNN inference become prohibitive for real-time applications.

Methodology: SimRec Architecture

SimRec utilizes a teacher-student architecture to "compress" graph knowledge into a non-graph model, allowing for efficient inference without the graph scaffold.

  • Teacher-Student Paradigm: A lightweight Graph Convolutional Network (GCN) teacher learns from the full interaction graph, while a simple, fast MLP student is trained to mimic its behavior.
  • Bi-Level Knowledge Distillation: The student learns through both prediction-level distillation (mimicking ranking scores) and embedding-level distillation (aligning hidden representations using InfoNCE contrastive loss).
  • Adaptive Contrastive Regularization: This is the framework's core novelty. SimRec uses a gradient-based recalibration module to detect when the teacher provides over-smoothed or noisy signals. If the distillation task contradicts the primary recommendation task, the model adaptively adjusts weights to ensure user embeddings remain distinct and diverse.

Key Insights: Embedding Uniformity

A striking insight from the research is that the graph is a training scaffold, not a permanent requirement. By visualizing the latent space, the authors demonstrate that SimRec produces a more uniform spread of users compared to traditional GNNs.

Distribution of learned user embeddings. SimRec achieves a more uniform spread compared to GNN-based baselines like LightGCN and HCCF, better preserving individual user preferences.

This uniformity indicates that SimRec effectively avoids over-smoothing, maintaining the unique preference signals of individual users rather than clumping them together in the latent space.

Key Results: Accuracy and Scalability

SimRec consistently outperformed state-of-the-art GNN models, including LightGCN, SGL, and HCCF, across standard benchmarks like Gowalla, Yelp, and Amazon.

Performance (Recall@20) vs. inference time on Amazon and Gowalla datasets. SimRec (Ours) achieves superior accuracy with significantly lower latency compared to GNN-based competitors.
  • Superior Accuracy: On the Amazon dataset, SimRec improved Recall@20 by over 10% compared to the best baseline.
  • Massive Speedup: On the large-scale Tmall dataset (40 million records), SimRec was approximately 2x faster in inference time than competitive GNN models (785s vs 1331s for SGL) while maintaining higher accuracy.
  • Robustness: Using the Mean Average Distance (MAD) metric, the authors confirmed that SimRec maintains significantly higher embedding diversity than traditional GNNs, successfully mitigating over-smoothing.

Conclusion

SimRec proves that the high-order relational insights of GNNs can be successfully distilled into lightweight models that do not require a graph during inference. By marrying knowledge distillation with adaptive contrastive learning, SimRec offers a scalable, robust solution for modern recommender systems. Future work will explore extending this graph-less approach to dynamic and temporal recommendation settings.

Summary updated

Abstract

Graph neural networks (GNNs) have shown the power in representation learning over graph-structured user-item interaction data for collaborative filtering (CF) task. However, with their inherently recursive message propagation among neighboring nodes, existing GNN-based CF models may generate indistinguishable and inaccurate user (item) representations due to the over-smoothing and noise effect with low-pass Laplacian smoothing operators. In addition, the recursive information propagation with the stacked aggregators in the entire graph structures may result in poor scalability in practical applications. Motivated by these limitations, we propose a simple and effective collaborative filtering model (SimRec) that marries the power of knowledge distillation and contrastive learning. In SimRec, adaptive transferring knowledge is enabled between the teacher GNN model and a lightweight student network, to not only preserve the global collaborative signals, but also address the over-smoothing issue with representation recalibration. Empirical results on public datasets show that SimRec archives better efficiency while maintaining superior recommendation performance compared with various strong baselines. Our implementations are publicly available at: https://github.com/HKUDS/SimRec.
BibTeX
@inproceedings{xia2023graph,
  author = {Lianghao Xia and Chao Huang and Jiao Shi and Yong Xu},
  title = {{Graph-less Collaborative Filtering}},
  booktitle = {The Web Conference},
  year = {2023},
  doi = {10.1145/3543507.3583196},
}
Figure