Foundation models excel at general tasks, but they often fall short when confronted with highly subjective or localized human preferences. A software engineer requires a model that understands their codebase's unique dialect; a patient needs a medical assistant tailored to their specific history; an author wants a creative writing partner that mimics their personal idiolect.
The naive solution to this problem is individual fine-tuning: training and hosting a dedicated copy of a model for every user. However, as a platform scales to millions of users, this approach triggers a severe infrastructure crisis. Full fine-tuning creates an unsustainable $O(N)$ storage bottleneck, where the deployment footprint scales linearly with the user base.
Even Parameter-Efficient Fine-Tuning (PEFT) techniques like LoRA, which reduce storage costs by isolating updates to small adapters, fail to solve the inference throughput bottleneck. In a production environment, high throughput relies on batching—processing multiple user requests simultaneously through parallel matrix multiplications on a GPU. As Yeming Wen demonstrates in Batched Low-Rank Adaptation of Foundation Models, standard serving frameworks assume that every request in a batch shares identical weights. When every user requires a distinct adapter, batching breaks down, forcing systems to choose between slow sequential processing or massive computational overhead.
To make personalization viable at scale, researchers must decouple a model's capacity for personal adaptation from the need for per-user model instances, as framed by Matthias De Lange et al. This article explores the core strategies emerging to bypass this parameter overhead bottleneck.
Test-Time Realignment and Weight-Free Steering
Instead of modifying a model's internal weights, one prominent direction alters the model's trajectory through its activation space dynamically during inference. Because the underlying parameters remain completely static, a single base model can serve thousands of distinct users simultaneously without breaking standard GPU batching operations.
This can be achieved by treating alignment as an online optimization problem solved at the moment of generation. For example, Songyang Gao et al. derive a closed-form solution for policy optimization that allows training-free alignment in a single inference step. Building on this intuition, Zhaowei Zhang et al. introduce Amulet, a framework that treats the generation of every single token as an independent online learning problem. By calculating a closed-form mathematical adjustment for the token probabilities on the fly, it realigns the model to user-specific preferences without incurring any weight-update overhead.
Standard Inference: [Input] ───> [Static Base Model] ───> [Standard Output]
│
Test-Time Steering: [Input] ───> [Static Base Model] ───> [Adjusted Output]
▲
[Closed-Form Shift /
Activation Steering]
Other methods manipulate the output distributions or internal activations directly. Jasper Dekoninck et al. demonstrate this via Language Model Arithmetic, combining the output distributions of different base and guidance models mathematically to steer text generation without per-attribute fine-tuning. Similarly, Minjun Zhu et al. propose Personality Activation Search (PAS), an activation-steering method that bypasses traditional alignment loops like DPO or PPO entirely, tailoring the model's persona in a fraction of the time by shifting internal hidden states during the forward pass.
Disentangling Preference from Core Generation
Another highly scalable strategy is to isolate user-specific traits into external structures or context windows, keeping the heavy foundation model entirely generic and untouched.
Bei Li et al. explicitly formalize this via architectural disentanglement. Their framework separates the user's preference representation from the primary text generation architecture. By applying optimization techniques to a low-dimensional latent encoder rather than the primary model parameters, the marginal cost of adding a new user profile drops to nearly zero, making millions of concurrent user profiles computationally feasible.
Disentangled Architecture:
┌─────────────────────────┐
[User Profile] ──>│ Low-Dim Latent Encoder │──┐
└─────────────────────────┘ │
▼
┌─────────────────────────┐ ┌──────────────────┐
[User Prompt] ──>│ Static Base Model │────>│ Tailored Output │
└─────────────────────────┘ └──────────────────┘
Alternatively, the context window itself can act as the personalization layer if the model is trained to generalize effectively across diverse instructions. Seongyun Lee et al. demonstrate this with JANUS, a model trained to adapt to thousands of distinct preferences purely through system message generalization. Rather than training individual reward models or adapters, a single static model dynamically alters its behavior based on highly detailed system prompts.
To avoid bloating the prompt and consuming valuable context tokens, user context can also be injected directly into the model's attention mechanism. Binxiao Xu et al. introduce Jarvis, a training-free framework that precomputes "memory" states for unique user metadata and retrieves them to inject user-specific information directly into the model's KV-Cache. This bypasses fine-tuning entirely while achieving significant throughput improvements over traditional long-prompt engineering.
Ultra-Lightweight Parameter Footprints and Shared Adapters
When direct parameter modification is unavoidable to capture complex nuances, scalability demands minimizing the per-user parameter footprint to the absolute physical limit, or sharing adapter architectures across large groups of users.
An extreme example of parameter minimization is demonstrated by Paul Michel et al. in the context of neural machine translation. Instead of modifying high-dimensional weight matrices, the authors adapt only the bias terms in the output softmax layer. This tiny modification successfully captures speaker-specific dialects and traits while preventing the memory footprint from scaling uncontrollably.
In speech and generative modeling, normalization layers have emerged as a highly efficient locus for personalization:
- AdaSpeech: Mingjian Chen et al. utilize a conditional layer normalization strategy for text-to-speech synthesis, reducing the unique storage footprint to just ~5KB per new speaker.
- GAN Fingerprinting: Jianwei Fei et al. insert a personalized normalization layer to generate unique, traceable model instances for thousands of users in milliseconds without full retraining.
- Acoustic Modeling: Taesup Kim et al. introduce Dynamic Layer Normalization (DLN), dynamically generating normalization parameters based on the input audio itself to adapt to speakers in real-time without increasing model size.
Where individual adapters are required, they can be designed to be shared or multi-tenant. Zhenlong Dai et al. address multi-user code personalization with MPCoder. Instead of assigning a unique adapter to every developer, they employ a single multi-user style adapter paired with low-dimensional, shared style vectors. This architecture achieves high stylistic similarity across thousands of individuals with a negligible increase in overall parameter count.
Similarly, Siyan Zhao introduces Group Preference Optimization (GPO), which meta-learns an independent, lightweight transformer module to predict group-specific rewards. This allows few-shot alignment to distinct interest groups or individuals, bypassing the massive data and compute requirements of traditional individual reinforcement learning loops.