Abstract
Integrating Large Language Models (LLMs) into Recommendation Systems (RS) often faces a "modality gap" between discrete user/item IDs and natural language. RA-Rec addresses this by introducing an ID Representation paradigm that aligns pre-trained ID embeddings with the LLM's latent space. By employing a lightweight, layer-specific projection module and contrastive learning, RA-Rec achieves superior recommendation performance with significantly fewer trainable parameters and less data than traditional fine-tuning methods.
Introduction
The integration of LLMs into recommendation systems typically follows two main approaches: direct ID usage or ID-to-text translation. Direct usage treats IDs as tokens, which lacks semantic depth, while translation converts IDs into descriptions, often exceeding context window limits and losing collaborative signals. RA-Rec proposes a third way: treating ID embeddings as "soft prompts" that can be understood by the LLM.
Methodology: The RA-Rec Framework
RA-Rec (Representation Alignment for Recommendation) bridges the gap between collaborative filtering and language modeling through several key components:
Hybrid Prompting and Layer-Specific Reparameterization
The framework combines "hard prompts" (textual instructions) with "soft prompts" derived from pre-trained ID embeddings. RA-Rec prepends learnable "virtual instructions" to these embeddings to guide the LLM. Furthermore, it uses a layer-specific projection module to transform ID embeddings for every transformer layer, acknowledging that different layers capture varying levels of semantic abstraction.
Alignment via Contrastive Learning
To ensure the ID embeddings are compatible with the LLM’s textual space, RA-Rec employs an InfoNCE-based contrastive loss. This aligns the transformed representations with the LLM's internal features. Crucially, the LLM backbone remains frozen, with only the alignment module being trained, making the process incredibly efficient.
Experimental Results
Evaluated on Amazon Books and Clothing datasets, RA-Rec demonstrates significant advantages:
- Accuracy: Achieves up to a 3.0% absolute improvement in HitRate@100 over state-of-the-art baselines.
- Efficiency: Uses approximately 40,000x fewer parameters than full fine-tuning and converges in 50% less time.
- Data Quality: Training on a curated 6% "Efficient" subset outperformed training on the full dataset, highlighting the importance of data denoising and diversity in alignment tasks.
Conclusion
RA-Rec provides a scalable and efficient solution for leveraging LLMs in recommendation. By aligning behavioral ID embeddings with textual reasoning capabilities, it preserves the strengths of both worlds while drastically reducing the computational overhead of model adaptation.
Summary updated