Optimizing Recall in Deep Graph Hashing Framework for Item Retrieval (Student Abstract)

Fangyuan Luo, Jun WuAAAI 2024 · DOI

Abstract

Hashing-based recommendation systems often suffer from a "metric mismatch" where models are trained using ranking objectives but evaluated on retrieval recall. This paper introduces SRLoss, a smooth recall loss that directly optimizes for recall, and the Approximation-Adjustable Gradient Estimator (AAGE) to address the non-differentiability of discrete hash codes. Experimental results on benchmark datasets demonstrate that this approach significantly outperforms state-of-the-art hashing methods in item retrieval accuracy.

Introduction

In modern large-scale recommender systems, retrieval is a critical first step, narrowing millions of items down to a manageable candidate set. While Embedding-based Retrieval (EBR) is standard, its high computational cost in real-valued space has led to the rise of Hashing-based Recommendation (HR). HR maps users and items into Hamming space (binary codes), enabling lightning-fast bitwise similarity calculations.

However, HR faces two primary challenges:

  1. Objective Mismatch: Most models optimize for Bayesian Personalized Ranking (BPR), while evaluation typically focuses on Recall. This discrepancy leads to suboptimal performance.
  2. Optimization Difficulty: The discrete nature of hash codes makes them non-differentiable. Standard workarounds like Straight-Through Estimation (STE) often provide inaccurate gradients, hindering model convergence.

Methods

The proposed framework builds upon the LightGCN architecture, incorporating two major innovations to bridge the gap between training objectives and evaluation metrics.

Overview of the proposed framework featuring SRLoss and AAGE.

Smooth Recall Loss (SRLoss)

Standard Recall formulas rely on non-differentiable indicator functions. To enable backpropagation, the authors replace these "hard" functions with smooth, differentiable approximations such as sigmoid or softplus. This allows the model to treat the Recall metric itself as the optimization target, resolving the objective mismatch.

Approximation-Adjustable Gradient Estimator (AAGE)

To handle the discrete sign function used in hashing, the authors introduce a "sign-swish" function for the backward pass. This estimator includes a temperature parameter ($\beta$) that controls the precision of the gradient approximation. Unlike the simplistic STE, AAGE provides a more accurate optimization direction, facilitating stable and accurate learning of binary representations.

Experiments

The model was evaluated on two benchmark datasets:

  • Gowalla: A location-based social network dataset (29,858 users, 40,981 items).
  • Yelp2018: A business review dataset (31,831 users, 40,841 items). Datasets were split into training, validation, and test sets using an 8:1:1 ratio.

Results

The proposed method significantly outperformed existing state-of-the-art hashing methods, including BGCH, HashGNN, and HashRec, across all evaluated metrics.

Performance comparison of the proposed method against state-of-the-art baselines on Gowalla and Yelp2018 datasets.

As shown in the results table, the proposed framework achieved a Recall@50 of 0.23082 on Gowalla and 0.10140 on Yelp2018, representing substantial improvements over the previous best-performing model, BGCH. Furthermore, improvements in Normalized Discounted Cumulative Gain (NDCG) indicate that the model not only retrieves more relevant items but also ranks them more effectively within the top-N results.

Conclusion

By directly optimizing for the evaluation metric and refining gradient estimation for discrete codes, this framework closes the accuracy gap between hashing and real-valued embeddings while maintaining high efficiency. Future work will focus on hyperparameter sensitivity analysis, particularly regarding the temperature parameter $\beta$, and exploring the scalability of the framework to industrial-scale datasets.

Summary updated

BibTeX
@inproceedings{luo2024optimizing,
  author = {Fangyuan Luo and Jun Wu},
  title = {{Optimizing Recall in Deep Graph Hashing Framework for Item Retrieval (Student Abstract)}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2024},
  doi = {10.1609/aaai.v38i21.30477},
}
Figure