CURP: Codebook-based Continuous User Representation for Personalized Generation with LLMs

Liang Wang, Xinyi Mou, Xiaoyou Liu, Xuanjing Huang, zhongyu wei

TL;DR

CURP introduces a plug-and-play framework for personalizing Large Language Models (LLMs) by mapping user history into a discrete prototype codebook. By separating long-term stable preferences from short-term query-aware dynamics, it achieves state-of-the-art personalization with just 28M trainable parameters (~0.36% of the base model size), bypassing the need for context-bloating prompts or expensive per-user fine-tuning.

What Problem Does This Paper Address?

Simulating individual user behavior and preferences is crucial for building effective dialogue systems, recommendation engines, and personalized content generators. Currently, LLM personalization follows two main paradigms, both of which suffer from a fundamental tension between efficiency and fidelity:

  1. Personalized Prompt-based Methods: These inject raw user interaction histories or text-based personas directly into the prompt. While lightweight and model-agnostic, raw text histories introduce significant noise, expand the context window length (increasing inference costs), and often cause the model to rely on shallow, stereotyped patterns.
  2. Personalized Parameter-based Methods: These fine-tune model weights (or user-specific LoRA modules) on an individual's data. While highly accurate, training and maintaining separate parameters for millions of users is computationally prohibitive and practically unscalable.
Different paradigms for user response simulation

The authors introduce CURP (Codebook-based Continuous User Representation) to bridge this gap, aiming for a robust, scalable, and highly accurate user simulation framework.

Methods

CURP's architecture is grounded in Self-Categorization Theory from psychology, which posits that identity is formed through a context-sensitive blend of shared social prototypes (e.g., #Teenager, #SportFan) rather than a fixed, isolated concept.

The framework consists of four key components operating in a two-stage training pipeline:

  • User Encoder: A pretrained, encoder-only model (Contriever) with bidirectional attention distills raw user history into dense tensor representations.
  • Prototype Codebook: To strip away surface-level textual noise, dense embeddings from a large behavioral pool are quantized using Product Quantization (PQ) and initialized via a Balanced K-Means strategy. This maps diverse user expressions into discrete, shared semantic prototypes.
  • Dual-Aggregator: User preferences are split into two complementary branches to prevent information entanglement:
  • Dynamic Branch: Uses Multi-Head Cross-Attention (MHCA) to fuse user prototypes with the current task query, capturing context-dependent intent.
  • Stable Branch: Uses a multi-layer Q-Former with shared learnable queries to compress historical prototypes into a fixed-length vector, capturing long-term intrinsic traits.
  • Decoder LLM: The aggregated stable and dynamic representations are projected via an MLP into the embedding space of a frozen decoder LLM (Qwen-2.5-7B-Instruct) to guide personalized generation.
The overall framework of our proposed CURP

Datasets

The authors evaluated CURP using the LaMP (Language Model Personalization) benchmark, specifically focusing on six publicly available tasks spanning classification and text generation:

  • LaMP-1: Citation Identification (Classification)
  • LaMP-2: Movie Tagging (Classification)
  • LaMP-3: Product Rating (Regression/Classification)
  • LaMP-4: News Headline Generation (Generation)
  • LaMP-5: Scholarly Title Generation (Generation)
  • LaMP-7: Tweet Paraphrasing (Generation)

Note: LaMP-6 (Email Subject Generation) was excluded due to privacy considerations.

Key Insights and Ideas

  • Quantization as a Denoiser: By forcing continuous embeddings into discrete codebook indices, CURP discards superficial lexical variations (like typos or wording changes) while retaining core semantic intent. This makes the model exceptionally robust against profile paraphrasing.
  • Disentangling Stable vs. Dynamic Traits: Combining long-term preferences and immediate query context into a single vector reduces personalization quality. Separating them allows the model to understand when a user's global behavior should override local context, and vice versa.
  • True Personalization vs. Shallow Copying: Case studies revealed that retrieval-based baselines (like BM25 or In-Context Learning) personalize by simply copying verbatim patterns from the prompt. CURP, by contrast, internalizes the user's overall preference distribution, allowing it to accurately predict behavior even when the query doesn't match any historical item closely.

Key Results

  • State-of-the-Art Performance: CURP outperformed strong prompt-based and parameter-based baselines on more than 70% of the evaluation metrics across the LaMP benchmark.
  • Parameter Efficiency: CURP achieves its results with only ~28M trainable parameters (about 0.36% of the total model size), matching or exceeding the performance of heavier parameter-per-user models like PROPER (which requires 8M parameters per user).
  • Robustness to Paraphrasing: When user histories were rewritten using an LLM to alter wording while preserving meaning, CURP maintained the highest consistency in generation quality compared to all baselines, proving its reliance on semantic concepts over surface text.

Limitations and Future Work

  • Lack of Explicit Personas: The framework relies entirely on behavioral interaction histories and lacks the ability to integrate direct textual persona descriptions (e.g., "I am a 30-year-old engineer").
  • Cross-Domain Constraints: The evaluation is restricted to single-domain tasks due to current dataset limitations; real-world users exhibit cross-domain behaviors that span multiple platforms.
  • Loss of Fine-Grained Lexical Habits: Because the model discards raw text in favor of semantic prototypes, it inevitably loses fine-grained behavioral patterns, such as a user's habitual word choices or unique slang.
  • Evaluation Metric Alignment: Standard automatic metrics like ROUGE sometimes penalize CURP when it successfully mimics a user's stylistic writing habits if those habits deviate from a single, un-numbered ground-truth reference.

Summary updated

Abstract

User modeling characterizes individuals through their preferences and behavioral patterns to enable personalized simulation with Large Language Models (LLMs) in contemporary approaches. However, existing methods, whether personalized prompt-based or parameter-based methods, face challenges in balancing personalization quality against computational and data efficiency. We propose a novel framework CURP, which employs a bidirectional user encoder, a discrete prototype codebook and a dual-aggregator to extract both multi-dimensional user traits and query-aware user characteristic. This design enables plug-and-play personalization with a small number of trainable parameters (about 28M parameters, about 0.36% of the total model size). Through extensive experiments, we show that CURP achieves superior performance and generalization compared to strong baselines on the language model personalization (LaMP) benchmark, while offering better interpretability and robustness.
BibTeX
@misc{wang2026curp,
  author = {Liang Wang and Xinyi Mou and Xiaoyou Liu and Xuanjing Huang and zhongyu wei},
  title = {{CURP: Codebook-based Continuous User Representation for Personalized Generation with LLMs}},
  year = {2026},
  eprint = {2602.00742},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2602.00742},
}
Figure