Abstract
Deep collaborative filtering (CF) models have significantly improved recommendation accuracy but are often hindered by high inference latency, particularly in real-time applications with large item catalogs. This paper proposes Hierarchical Decision Networks (HDNs), a novel architecture that employs an "early exit" strategy to accelerate inference. HDNs consist of stacked decision modules that learn to reject irrelevant items at early layers, thereby bypassing redundant computations. To optimize performance, we introduce a distillation-based training process where the HDN learns to mimic a complex teacher model's rankings while establishing efficient rejection thresholds. Evaluations on standard benchmarks, including MovieLens and Pinterest, show that HDNs provide substantial inference speedups without compromising recommendation quality, often outperforming traditional two-stage ranking pipelines.
Introduction
Collaborative Filtering (CF) is a cornerstone of modern personalized recommendation systems. While deep learning has pushed the boundaries of CF accuracy, it has introduced a massive inference bottleneck. To generate a recommendation list, a deep CF model typically calculates a relevance score for every item in a catalog for a specific user. For a model with many layers, this requires a full forward pass for every user-item pair. In production environments with millions of items, this $O(\text{layers} \times \text{items})$ complexity makes real-time, end-to-end neural recommendations prohibitively slow.
The HDN Architecture
The core innovation of this work is the Hierarchical Decision Network (HDN). Instead of a monolithic deep network, an HDN consists of $N$ stacked Decision Modules followed by a final Scoring Module.
Each decision module takes the current user-item representation and decides whether to "reject" the item or pass it to the next, deeper layer. If an item is not rejected, the module produces a more refined "interaction embedding" for the subsequent layer. This mechanism ensures that items clearly irrelevant to a user are dropped after just one or two layers, drastically reducing the total computational cost. Theoretically, if the rejection rate is $\ge 0.5$, the inference cost becomes independent of the network's depth, reducing complexity from $O(\text{Depth})$ to $O(1)$ per item.
Distillation-Based Training
Training a network to "know when to quit" is challenging. The authors employ a Knowledge Distillation framework:
- Teacher-Student Model: A high-performing "Teacher" network (e.g., NeuMF or ConvNCF) is first trained.
- Learning to Rank: The HDN (the "Student") is trained to mimic the Teacher’s final scores using KL-divergence.
- Learning to Reject: The Student uses the Teacher’s rankings to define rejection thresholds. If the Teacher ranks an item poorly, the Student learns to reject it in its early modules using a specialized hinge loss function.
Experimental Results
The HDN was evaluated on four benchmark datasets: MovieLens (1M and 20M), Pinterest, and Yelp.
As shown in the results, HDNs demonstrate dramatic reductions in inference time, particularly in memory-constrained scenarios with small batch sizes. Because the "item pool" shrinks at each successive layer, the computational burden of deeper layers is significantly mitigated. Crucially, HDNs maintained or exceeded the Hit Ratio (HR) and Normalized Discounted Cumulative Gain (NDCG) of state-of-the-art models like NeuMF and ConvNCF across all datasets. This suggests that the hierarchical filtering process may act as a form of regularization, reducing noise by preventing the model from overfitting on clearly irrelevant user-item pairs.
Conclusion
Hierarchical Decision Networks offer a robust solution to the depth-latency tradeoff in deep collaborative filtering. By integrating an early-exit mechanism directly into the neural architecture and leveraging knowledge distillation, HDNs achieve high-speed inference without sacrificing recommendation accuracy. Future work may explore more complex decision logic, such as attention mechanisms, and the integration of content-based metadata to further enhance filtering performance.
Summary updated