Learning Binarized Representations with Pseudo-positive Sample Enhancement for Efficient Graph Collaborative Filtering

Yankai Chen, Yue Que, Xinni Zhang, Chen Ma, Irwin King

Recommender systems often rely on massive, memory-heavy embeddings, making real-time inference computationally expensive. This paper introduces BiGeaR++, a framework that compresses these embeddings into 1-bit binary codes without sacrificing accuracy. By leveraging "pseudo-positive" samples and a novel distillation process, BiGeaR++ achieves over 8x faster inference and massive memory savings while matching—and sometimes exceeding—the performance of full-precision models.

The Challenge of Embedding Binarization

Modern recommender systems heavily rely on Graph Collaborative Filtering (GCF) to match users with items. These models learn continuous, high-dimensional vector representations (embeddings) for every user and item. As platforms grow, storing and computing the inner products of these dense floating-point embeddings becomes a massive bottleneck for real-time online inference.

A promising solution is representation binarization (or 1-bit quantization), which converts these floating-point numbers into simple binary codes (+1 or -1). This allows systems to replace slow floating-point math with lightning-fast bitwise operations. However, simply snapping values to +1 or -1 causes severe information loss, destroying the model's ability to accurately rank items. Previous binarization methods focused mostly on the math of quantization, neglecting the resulting loss of ranking capability and the inaccurate gradient estimations that occur during training.

Overview of BiGeaR++ and comparison of target embedding tables across different model types.

The BiGeaR++ Framework

BiGeaR++ builds on a previous model (BiGeaR) and introduces a suite of techniques to make 1-bit embeddings just as expressive as full-precision ones. The core innovations focus on how the model learns from "pseudo-positive" signals to preserve ranking capability.

The detailed workflow of BiGeaR++, illustrating the stages of information aggregation, embedding quantization, and model training with pseudo-positive enhancement.

Core Innovations

  • Dual Inference Distillation: BiGeaR++ uses a teacher-student framework. A full-precision "teacher" model guides the binarized "student" model. Instead of standard knowledge distillation, BiGeaR++ forces the student to mimic the teacher's ranking order. Crucially, it distills rankings not just for known positive interactions, but also for pseudo-positive items—real items the user hasn't interacted with, but that the teacher model highly recommends.
  • Pseudo-positive Sample Synthesis: Standard models learn by contrasting positive items against random negative items. BiGeaR++ synthesizes "hard" fake negative embeddings directly in the latent space by mixing features of positive items with the hardest negative items. This forces the binarized model to learn much sharper decision boundaries.
  • Graph Layer-wise Quantization: Instead of just binarizing the final output, BiGeaR++ binarizes the intermediate embeddings at every layer of the graph convolution. This feature magnification effectively combats the drop in expressivity usually caused by 1-bit compression.
  • Accurate Gradient Estimation: Standard binarization uses the non-differentiable sign() function. BiGeaR++ uses a smooth approximation of the Dirac delta function to calculate highly accurate gradients during training, avoiding the flaws of the standard Straight-Through Estimator (STE).
  • Bitwise Acceleration: During inference, the model replaces floating-point inner products with XNOR and Popcount bitwise operations, drastically speeding up predictions.

Experimental Evaluation

The authors evaluated their model on five standard real-world benchmark datasets: MovieLens (1M), Gowalla, Pinterest, Yelp2018, and Amazon-Book.

Key Results

  • Massive leap over binarized baselines: BiGeaR++ significantly outperforms state-of-the-art binarized models (like HashGNN), improving Recall@20 by 32.47%–54.49% and NDCG@20 by 25.46%–50.86%.
  • Full-precision Parity: Despite being compressed to 1-bit, BiGeaR++ achieves 98.11%–108.26% of the recommendation capability of LightGCN (the best-performing full-precision model). In some cases, it actually outperforms the uncompressed model due to regularization effects.
  • Hardware Efficiency: On the MovieLens dataset, BiGeaR++ reduced the memory footprint of the embedding table from 9.79MB to just 1.08MB (a ~9x reduction). Online inference is roughly 8x faster than LightGCN.

Insights and Future Work

The success of BiGeaR++ highlights that in highly constrained 1-bit spaces, learning from random negative samples is insufficient. Synthesizing "hard" pseudo-positive embeddings and distilling relative ranking order are crucial for capturing nuanced user preferences that would otherwise be lost in quantization.

While effective, the authors noted that trying to make embedding "scalers" learnable via stochastic optimization failed to improve performance. Additionally, synthesizing pseudo-positive samples adds computational overhead during the training phase. Future work may explore the integration of Large Language Models (LLMs) and multimodal information to further enrich binarized representations.

Summary updated

BibTeX
@article{chen2025learning,
  author = {Yankai Chen and Yue Que and Xinni Zhang and Chen Ma and Irwin King},
  title = {{Learning Binarized Representations with Pseudo-positive Sample Enhancement for Efficient Graph Collaborative Filtering}},
  journal = {ACM Trans. Information Systems},
  year = {2025},
  doi = {10.1145/3744239},
}
Figure