TL;DR
RACE is a novel framework for efficient on-device sequential recommendation that dynamically adjusts model capacity and input data based on the complexity of each user's intent. By using a "ranking-uncertainty" metric to guide instance-specific slimming, it achieves significantly higher accuracy and a ~2.4x reduction in computational cost without requiring expensive on-device retraining.
What Problem Does This Paper Address?
Sequential recommendation systems—the engines that predict what you'll click next based on your history—are increasingly moving from the cloud to our mobile devices to improve privacy and reduce latency. However, mobile devices have limited battery and processing power.
Current "efficient" models typically use a "one-size-fits-all" compression strategy. They prune the model to a fixed size regardless of whether a user's intent is simple (e.g., someone who only buys sci-fi books) or complex (e.g., someone with volatile, shifting interests). This leads to a Capacity-Complexity Mismatch: we waste precious compute on easy cases and provide poor recommendations for hard ones.
The authors also show that simple heuristics, like how popular a user's history is, don't actually correlate well with how "hard" a recommendation is for the model.
Methods
The authors propose RACE (Ranking-uncertainty Aware Capacity-Adaptive Customization), which treats model uncertainty as a proactive resource allocator.
- Ranking-Uncertainty Metric: Instead of just looking at the confidence of a single item, RACE measures the stability of the entire top-$K$ list. It uses a Variational Bayesian last layer to estimate the probability that one item will outrank another. If the predicted list is "shaky" (high uncertainty), the instance is deemed complex.
- Early Anticipation: To save compute, the model can't wait until the end to decide how much to prune. RACE uses a lightweight Threshold Predictor that looks at early-layer features to "guess" the final uncertainty and set pruning ratios for the rest of the network.
- Dual-Space Slimming:
- Model Structure: A hypernetwork-style "Sensitivity Extractor" identifies which neural pathways (channels) are most relevant to the current user's intent and deactivates the rest.
- Input Data: "Progressive Sequence Filtering" uses attention scores to identify and drop redundant or noisy items from the user's history block-by-block.
- Intrinsic Feature Reuse: Crucially, RACE doesn't use heavy external models to decide what to prune. It reuses the model's own internal hidden states, making the customization process itself extremely efficient.
Datasets
The framework was evaluated on four diverse, real-world datasets commonly used in recommendation research:
- MovieLens-100k: Movie ratings and history.
- Douban: A Chinese social networking service for books, movies, and music.
- Amazon-CD: Music CD purchase histories.
- Amazon-Game: Video game purchase histories.
Key Insights and Ideas
- Uncertainty as a Proxy for Complexity: The core "aha" moment is that a model's internal "confusion" about a ranking is the most reliable signal for how much computational effort it should expend.
- List-wise vs. Point-wise: Most uncertainty methods look at individual items. RACE recognizes that recommendation is a ranking task, so it measures the stability of the relative order of items.
- Joint Optimization: By slimming both the model parameters and the input data simultaneously, RACE finds a much better efficiency-accuracy trade-off than methods that only look at one side.
Key Results
RACE was tested against state-of-the-art baselines using two popular backbones, SASRec and BSARec.
- Accuracy Boost: On the SASRec backbone, RACE improved NDCG (a measure of ranking quality) by an average of 40.87% across all datasets.
- Efficiency Gains: It achieved these gains while reducing FLOPs (floating-point operations) by 2.37x to 2.49x.
- Real-world Latency: On a 4-core CPU, RACE reduced inference latency by over 40% compared to standard on-device finetuning methods.
- Adaptability: When tested under stricter-than-normal resource constraints, RACE "gracefully" degraded, maintaining high accuracy while automatically becoming more aggressive with its pruning.
Limitations and Future Work
While RACE is highly effective for Transformer-based sequential models, the authors note a few areas for expansion:
- Generative Recommenders: Future work could extend this uncertainty-aware logic to Large Language Model (LLM) based recommenders, perhaps using uncertainty to decide how many tokens to generate or how deep to search.
- Model Refreshing: The ranking-uncertainty signal could also be used to decide when a model has become stale and needs a full retraining from the cloud.
- Diverse Architectures: While tested on Transformers, the general principle of uncertainty-driven capacity could potentially be applied to other architectures like GNNs or State Space Models (SSMs).
Summary updated