Abstract
Standard Large Language Models (LLMs) are typically designed as "universal" systems, yet they often struggle with subjective tasks where "ground truth" varies by individual. This paper explores personalizing LLMs by fine-tuning them with simple User IDs, demonstrating performance boosts of up to 165% in tasks like emotion recognition and hate speech detection compared to non-personalized baselines.
Introduction: The Challenge of Subjectivity
Most NLP benchmarks operate under the assumption of a single, objective "correct" answer. However, human perception is inherently subjective; whether a comment is perceived as "sarcastic," "hateful," or "annoying" depends heavily on an individual's background and beliefs. While LLMs are powerful zero-shot reasoners, their "one-size-fits-all" nature ignores this variance. Existing personalization attempts often rely on In-Context Learning (ICL), which is limited by context window length and fails to update the model's internal weights.
Methodology
The researchers evaluated two primary strategies for personalization across several LLM architectures, including Mistral 7B, Phi-2, StableLM, and Flan-T5.
- Prompt-based Personalization (Few-shot): This involves including 1 or 2 examples of a specific user’s previous annotations directly in the prompt.
- Personalized Fine-tuning: This core contribution utilizes Parameter-Efficient Fine-Tuning (PEFT) via LoRA (Low-Rank Adaptation). Two setups were tested:
- CLS-P (Personalized Classification): A new classification head is added, and a "User ID" is injected into the input prompt to help the model learn specific labeling patterns.
- LM-P (Personalized Language Modeling): The model generates the label as text, conditioned on the User ID.
By using User IDs as a proxy for individual context, the model learns a latent representation of user perception without requiring extensive demographic data.
Datasets
The study utilized two major English-language datasets characterized by high annotator disagreement:
- GoEmotions: 58k Reddit comments annotated with 28 emotional categories.
- Unhealthy Conversations: 230k annotations on news comments categorized by attributes like "hostile" or "sarcastic."
The datasets were partitioned to test the models' ability to predict how a specific user would label a previously unseen piece of text.
Key Results and Insights
The results demonstrate a massive "personalization gain"—the percentage improvement over a non-personalized baseline.
- Significant Performance Boosts: On the Unhealthy Conversations dataset, StableLM 3B saw a 164.2% gain in F1-macro score when moving to personalized classification.
- Model Excellence: Mistral 7B achieved a 128.7% gain on Unhealthy Conversations and a 64.1% gain on GoEmotions using the CLS-P method.
The study found that simply prepending a User ID to a prompt during fine-tuning allows the model to effectively "switch" its perspective, suggesting that LLMs can store multiple subjective "worldviews" within their weights. Furthermore, for most open-source models, fine-tuning provided much larger gains than few-shot prompting.
Limitations and Future Work
While promising, the use of User IDs acts as a "black box" and does not explain the underlying reasons for specific labeling behaviors. Additionally, the "cold start problem" remains, as the method requires a history of annotations for a user before personalization can occur. Future research may focus on incorporating richer contextual factors and addressing the computational challenges of maintaining personalized adapters for large user bases.
Summary updated