TL;DR
HiGenTrans is a hierarchical framework that supercharges generative recommendation by injecting Large Language Model (LLM) knowledge at three distinct levels: input, internal representation, and final decision-making. By moving beyond shallow, input-only fusion, it achieves up to a 33.7% improvement in ranking accuracy (NDCG@10) while maintaining the efficiency required for industrial-scale deployment.
What Problem Does This Paper Address?
Generative recommendation (GenRec) is a rising paradigm that simplifies the traditional multi-stage pipeline (retrieval, ranking, reranking) into a single autoregressive task. Instead of scoring every item in a catalog, these models generate "Semantic IDs" (SIDs)—discrete codes representing items—token by token.
However, the authors identify two major bottlenecks in current GenRec systems:
- Shallow Semantic Fusion: Most systems only use LLM knowledge at the input level (e.g., as a prompt or prefix). This ignores the rich, hierarchical abstractions stored in the intermediate layers of a pretrained LLM.
- Ranking Mismatch: GenRec models typically use beam search to find the most likely token sequences. Unfortunately, the most probable sequence isn't always the most relevant item for the user.
HiGenTrans aims to bridge these gaps by creating a "deep" knowledge transfer that persists throughout the entire generation process.
Methods
The HiGenTrans framework introduces three interactive modules to transfer knowledge from a frozen LLM (like Qwen) to a generative recommendation backbone:
- Global Semantic Priming (GSP): At the input level, the system serializes a user’s interaction history and passes it through a frozen LLM. The resulting global representation is projected into the "Beginning of Sequence" (BOS) token, providing the generator with a semantic "prior" before it generates a single token.
- Hierarchical Representation Transfer (HRT): At the representation level, the authors map specific intermediate layers of the LLM to corresponding layers in the GenRec model. They use mean-centered projections and gated residual connections to align these two different latent spaces, ensuring that the collaborative backbone of the recommender is enriched by, rather than replaced by, semantic features.
- Multi-Signal Decision Calibration (MDC): At the decision level, a lightweight Mixture-of-Experts (MoE) calibrator reranks the candidates produced by beam search. It combines semantic signals (LLM-based relevance), generative signals (sequence likelihood), and contextual signals (how a candidate compares to others in the beam).
To make this feasible for production, the authors use a vector quantization-based compression scheme. This allows them to store high-dimensional LLM features as compact Semantic IDs offline, which are then reconstructed during online inference without needing to run the full LLM.
Datasets
The authors evaluated HiGenTrans on three public benchmarks from the Amazon review dataset:
- Beauty: 22k users, 12k items.
- Sports: 35k users, 18k items.
- Toys: 19k users, 11k items.
Additionally, they conducted evaluations on two large-scale industrial cross-domain datasets from Kuaishou to test the framework's real-world applicability.
Key Insights and Ideas
- Depth is Discriminative: The authors found that intermediate LLM layers (specifically around Stage 3 of the LLM hierarchy) are significantly better at distinguishing ground-truth items from negatives than the initial layers. This justifies the "hierarchical" approach of HRT.
- Likelihood $\neq$ Relevance: The success of the MDC module proves that raw beam search scores are insufficient. By looking at the "neighborhood" of candidates and their semantic consistency, the model can promote relevant items that were originally buried in the beam.
- Alleviating Sparsity: The semantic knowledge transfer is particularly effective for "long-tail" items (items with very few interactions) and users with short histories. When collaborative data is sparse, the LLM’s understanding of item metadata fills the gap.
Key Results
HiGenTrans consistently outperformed state-of-the-art baselines like TIGER and GRLM:
- Accuracy Gains: On the Toys dataset, HiGenTrans improved Recall@10 by 12.9% and NDCG@10 by 33.7% over the strongest baseline.
- Ranking Correction: The MDC module successfully promoted roughly 43% to 55% of ground-truth items from the bottom half of the candidate list (ranks 11-20) into the top 10.
- Industrial Impact: In real-world e-commerce scenarios, the framework showed massive improvements in SID prediction accuracy, particularly for coarse-grained item categories (up to 61.98 percentage points).
Limitations and Future Work
- Candidate Bottleneck: The MDC module can only rerank what the generator produces. If the generator fails to include a relevant item in its beam search results entirely, the calibrator cannot "rescue" it.
- Frozen LLM: While using a frozen LLM is efficient, the authors note that the semantic features are static. Future work could explore lightweight fine-tuning (like LoRA) specifically for the recommendation task.
- Tokenization Dependency: The performance is still tied to the quality of the Semantic IDs. Improving how items are mapped to discrete codes remains a parallel area for improvement.
Summary updated