Introduction
Personalizing Large Language Models (LLMs) to reflect individual user styles and preferences is a significant challenge. Current approaches typically rely on prompt-based methods like Retrieval-Augmented Generation (RAG), which can be computationally expensive and risk leaking raw data, or training-based methods like Parameter-Efficient Fine-Tuning (PEFT/LoRA), which often lack the "collaborative intelligence" found in larger communities.
PriME (Privacy-Preserving Model Merging via Evolutionary Algorithms) addresses these issues by merging specialized modules from a community of users. It utilizes gradient-free evolutionary optimization to maximize task performance while explicitly minimizing privacy risks, protecting contributors from membership inference attacks (MIA) without sacrificing personalization.
Methodology: The PriME Framework
PriME treats personalization as a multi-objective optimization problem. The process involves three main stages:
- User Selection: The system identifies the "Top-K" most similar sharing users from a pool based on text embeddings of their history.
- Evolutionary Merging: Instead of simple weight averaging, PriME employs Evolutionary Strategies (specifically CMA-ES) to find optimal interpolation weights for the shared LoRA modules. This gradient-free approach allows for the direct optimization of non-differentiable metrics such as ROUGE or accuracy.
- Privacy-Aware Objective: To protect sharing users, the optimization objective includes a penalty term for privacy leakage. This term minimizes the similarity between the merged model's predictions and those of individual contributors, creating a "blend" that captures general preferences without mimicking private data too closely.
Experimental Evaluation
The framework was evaluated using the LaMP (Language Model Personalization) benchmark, covering classification tasks (citation identification, movie tagging, product rating) and generation tasks (news headlines, scholarly titles, tweet paraphrasing). Experiments were conducted across several base models, including Llama-2-7b, Llama-3.1-8B, and Llama-3.2-3B.
Key Results and Insights
- Superior Performance: PriME consistently outperformed state-of-the-art baselines, achieving up to a 45% improvement in task performance on certain benchmarks. It often surpassed models trained solely on the target user's data, demonstrating the value of collaborative knowledge.
- Controllable Privacy-Utility Trade-off: By adjusting the coefficient $\alpha$, developers can balance performance against privacy. Higher $\alpha$ values reduce the risk of membership inference attacks by making the model's output less similar to any single contributor's training data.
- Efficiency: The evolutionary search is remarkably efficient, requiring only 20-60% of user history during optimization to achieve results comparable to using the full dataset.
- MIA Robustness: PriME significantly lowers the Area Under the Curve (AUC) for membership inference attacks compared to traditional merging methods, making it much harder for attackers to identify specific contributors.
Conclusion and Future Directions
PriME offers a robust framework for personalizing LLMs while maintaining user privacy. While it empirically reduces privacy risks, it does not yet provide formal mathematical guarantees like Differential Privacy (DP). Future work will explore integrating DP and resolving potential weight conflicts between users with highly divergent styles. Additionally, hybrid approaches combining PriME's merged parameters with prompt-based methods like RAG show promise for further enhancing personalization.
Summary updated