HiGenTrans: Hierarchical LLM Knowledge Transfer for Generative Recommendation

Zhuohang Jiang, Yuxin Chen, Haohao Qu, ZeFei luo, Yunlong Wang, Xu Yuan, Haojie Ding, Boheng Zhang, Fan Yang, Hongyang Wang, Jun Wang, Zhongbo Sun, Wenqi Fan, Li Qing, Wenwu Ou, Peng JiangarXiv 2026 · arXiv

TL;DR

HiGenTrans is a hierarchical framework that supercharges generative recommendation by injecting Large Language Model (LLM) knowledge at three distinct levels: input, internal representation, and final decision-making. By moving beyond shallow, input-only fusion, it achieves up to a 33.7% improvement in ranking accuracy (NDCG@10) while maintaining the efficiency required for industrial-scale deployment.

What Problem Does This Paper Address?

Generative recommendation (GenRec) is a rising paradigm that simplifies the traditional multi-stage pipeline (retrieval, ranking, reranking) into a single autoregressive task. Instead of scoring every item in a catalog, these models generate "Semantic IDs" (SIDs)—discrete codes representing items—token by token.

However, the authors identify two major bottlenecks in current GenRec systems:

  1. Shallow Semantic Fusion: Most systems only use LLM knowledge at the input level (e.g., as a prompt or prefix). This ignores the rich, hierarchical abstractions stored in the intermediate layers of a pretrained LLM.
  2. Ranking Mismatch: GenRec models typically use beam search to find the most likely token sequences. Unfortunately, the most probable sequence isn't always the most relevant item for the user.

HiGenTrans aims to bridge these gaps by creating a "deep" knowledge transfer that persists throughout the entire generation process.

Figure 1: Motivation and overview of HiGenTrans. Existing GenRec and LLM4Rec methods suffer from shallow seman- tic fusion and misalignment between generation likelihood and item relevance. HiGenTrans addresses these limitations through Global Semantic Prior (GSP), Hierarchical Represen- tation Transfer (HRT), and Multi-signal Decision Calibration (MDC), enabling richer semantic integration and more accu- rate recommendation ranking.

Methods

The HiGenTrans framework introduces three interactive modules to transfer knowledge from a frozen LLM (like Qwen) to a generative recommendation backbone:

  1. Global Semantic Priming (GSP): At the input level, the system serializes a user’s interaction history and passes it through a frozen LLM. The resulting global representation is projected into the "Beginning of Sequence" (BOS) token, providing the generator with a semantic "prior" before it generates a single token.
  2. Hierarchical Representation Transfer (HRT): At the representation level, the authors map specific intermediate layers of the LLM to corresponding layers in the GenRec model. They use mean-centered projections and gated residual connections to align these two different latent spaces, ensuring that the collaborative backbone of the recommender is enriched by, rather than replaced by, semantic features.
  3. Multi-Signal Decision Calibration (MDC): At the decision level, a lightweight Mixture-of-Experts (MoE) calibrator reranks the candidates produced by beam search. It combines semantic signals (LLM-based relevance), generative signals (sequence likelihood), and contextual signals (how a candidate compares to others in the beam).

To make this feasible for production, the authors use a vector quantization-based compression scheme. This allows them to store high-dimensional LLM features as compact Semantic IDs offline, which are then reconstructed during online inference without needing to run the full LLM.

Figure 2: Overview of HiGenTrans, a three-level semantic knowledge transfer framework for generative recommendation. Global Semantic Priming (GSP) incorporates global user semantics into the BOS token. Hierarchical Representation Transfer (HRT) injects multi-level LLM representations into the generative backbone. Multi-Signal Decision Calibration (MDC) employs MoE-based multi-signal calibration to refine candidate rankings.

Datasets

The authors evaluated HiGenTrans on three public benchmarks from the Amazon review dataset:

  • Beauty: 22k users, 12k items.
  • Sports: 35k users, 18k items.
  • Toys: 19k users, 11k items.

Additionally, they conducted evaluations on two large-scale industrial cross-domain datasets from Kuaishou to test the framework's real-world applicability.

Key Insights and Ideas

  • Depth is Discriminative: The authors found that intermediate LLM layers (specifically around Stage 3 of the LLM hierarchy) are significantly better at distinguishing ground-truth items from negatives than the initial layers. This justifies the "hierarchical" approach of HRT.
  • Likelihood $\neq$ Relevance: The success of the MDC module proves that raw beam search scores are insufficient. By looking at the "neighborhood" of candidates and their semantic consistency, the model can promote relevant items that were originally buried in the beam.
  • Alleviating Sparsity: The semantic knowledge transfer is particularly effective for "long-tail" items (items with very few interactions) and users with short histories. When collaborative data is sparse, the LLM’s understanding of item metadata fills the gap.

Key Results

HiGenTrans consistently outperformed state-of-the-art baselines like TIGER and GRLM:

  • Accuracy Gains: On the Toys dataset, HiGenTrans improved Recall@10 by 12.9% and NDCG@10 by 33.7% over the strongest baseline.
  • Ranking Correction: The MDC module successfully promoted roughly 43% to 55% of ground-truth items from the bottom half of the candidate list (ranks 11-20) into the top 10.
  • Industrial Impact: In real-world e-commerce scenarios, the framework showed massive improvements in SID prediction accuracy, particularly for coarse-grained item categories (up to 61.98 percentage points).
Figure 3: SID-based compression for hierarchical transfer. Multi-level LLM embeddings are compressed into SIDs and reconstructed for stage-wise semantic transfer.

Limitations and Future Work

  • Candidate Bottleneck: The MDC module can only rerank what the generator produces. If the generator fails to include a relevant item in its beam search results entirely, the calibrator cannot "rescue" it.
  • Frozen LLM: While using a frozen LLM is efficient, the authors note that the semantic features are static. Future work could explore lightweight fine-tuning (like LoRA) specifically for the recommendation task.
  • Tokenization Dependency: The performance is still tied to the quality of the Semantic IDs. Improving how items are mapped to discrete codes remains a parallel area for improvement.

Summary updated

Abstract

Generative recommendation promises to simplify conventional multi-stage recommender systems by generating item Semantic IDs (SIDs) autoregressively. However, two obstacles limit its effective- ness in industrial settings: interaction-only supervision underuses the semantic knowledge encoded in pretrained language models, while beam-search likelihood ranks token sequences rather than directly estimating item relevance. To address these limitations, we propose HiGenTrans, a three-level framework that introduces LLM knowledge at the input, representation, and decision levels. First, Global Semantic Priming (GSP) adds an LLM-derived user representation to the beginning-of-sequence embedding, provid- ing a semantic prior before SID generation. Second, Hierarchi- cal Representation Transfer (HRT) aligns intermediate LLM representations with depth-matched GenRec layers through mean- centered projections and gated residual connections. Finally, Multi- Signal Decision Calibration (MDC) uses a lightweight mixture- of-experts calibrator to combine semantic relevance, generation confidence, and candidate-set context when reranking beam-search outputs. All semantic features are computed only from the observ- able interaction prefix, preventing target leakage, and can be cached offline. Experiments on three public benchmarks show that HiGen- Trans improves Recall@10 by up to 12.9% and NDCG@10 by up to 33.7% over the strongest baselines. These results demonstrate the value of transferring LLM knowledge at complementary stages of the generative recommendation pipeline.
BibTeX
@misc{jiang2026higentrans,
  author = {Zhuohang Jiang and Yuxin Chen and Haohao Qu and ZeFei luo and Yunlong Wang and Xu Yuan and Haojie Ding and Boheng Zhang and Fan Yang and Hongyang Wang and Jun Wang and Zhongbo Sun and Wenqi Fan and Li Qing and Wenwu Ou and Peng Jiang},
  title = {{HiGenTrans: Hierarchical LLM Knowledge Transfer for Generative Recommendation}},
  year = {2026},
  eprint = {2611.01129},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.01129},
}
Figure