Abstract
Federated learning for Large Language Models (LLMs) faces significant challenges due to data heterogeneity across clients, leading to "client drift" and poor personalization. This paper introduces FedLoRA-Optimizer, a framework that decomposes Low-Rank Adaptation (LoRA) matrices into magnitude and direction components. By globally optimizing directional updates for better generalization and locally tuning magnitude for personalization, FedLoRA-Optimizer effectively mitigates client drift while enhancing model performance on diverse tasks.
Introduction
Fine-tuning Large Language Models (LLMs) is computationally intensive. While LoRA (Low-Rank Adaptation) reduces this burden by training small adapter matrices, its application in Federated Learning (FL) is hindered by non-IID (Independent and Identically Distributed) data. Standard FL methods often treat LoRA matrices as monolithic blocks, failing to account for the distinct roles different components play in learning. This results in a global model that struggles to find a middle ground for all clients, leading to poor generalization and insufficient personalization.
Methodology
FedLoRA-Optimizer adopts a surgical approach to weight updates by decomposing them into directional and magnitude components, inspired by Weight Normalization and DoRA.
Weight Decomposition
The weight updates are split into:
- Directional vectors ($V$): Determining the "angle" or nature of the update.
- Magnitude vectors ($m$): Determining the "strength" or scale of the update.
The Global-Local Pipeline
The framework utilizes a two-stage optimization pipeline:
- Global Optimization: The server aggregates the directional vectors of Matrix A from all participating clients. Empirical evidence suggests that Matrix A's direction carries critical shared knowledge essential for cross-task generalization.
- Local Optimization: Upon receiving the updated global model, clients perform a local fine-tuning step focused on the magnitude vectors of Matrix B. This allows individual clients to adjust the intensity of specific features to suit their local data without compromising the shared global structure.
Empirical Motivation: The Sensitivity Gap
The design of FedLoRA-Optimizer is rooted in the observation of a "Sensitivity Gap" between LoRA matrices A and B. Exploratory experiments revealed that the directional variation ($\Delta D$) in Matrix A is approximately 1.7x greater than in Matrix B, while the magnitude variation ($\Delta M$) in Matrix B is significantly higher—roughly 41x greater than in Matrix A.
This suggests a functional specialization where Matrix A acts as a "compass" (defining the task direction) and Matrix B acts as a "volume knob" (defining the application strength).
Experimental Results
The authors evaluated FedLoRA-Optimizer using LLaMA2-7B and DeepSeek-7B on the Databricks-Dolly-15k and Natural Instructions datasets. The framework consistently outperformed standard LoRA and other lightweight tuning methods.
- Generalization: Global performance improved by an average of 0.39%.
- Personalization: Local performance saw an average improvement of 0.59%.
- Task-Specific Gains: On Question Answering (QA) tasks using LLaMA2-7B, FedLoRA-Optimizer reached 66.69% accuracy, compared to 61.69% for standard LoRA.
- Ablation Success: The pipeline-structured approach (Global then Local) significantly outperformed non-pipeline alternatives, proving that the order of optimization is critical.
Conclusion
FedLoRA-Optimizer demonstrates that a nuanced approach to LoRA matrix updates can effectively balance global generalization with local personalization in federated settings. By leveraging the inherent sensitivity differences between magnitude and direction, the framework provides a robust solution to the challenges of data heterogeneity in LLM fine-tuning. Future work will focus on more extreme heterogeneity scenarios and further refining optimization strategies to boost performance deltas.
Summary updated