TL;DR
This paper introduces a novel Split Federated Learning (SFL) framework designed to efficiently fine-tune Large Language Models (LLMs) on resource-constrained, heterogeneous mobile devices. By strategically splitting LLM layers and optimizing the training order, the approach significantly reduces memory usage on both mobile devices and the server while maintaining comparable performance to existing methods.
What Problem Does This Paper Address?
Large Language Models (LLMs) are becoming increasingly powerful, but fine-tuning them for specific tasks on individual devices is challenging due to their substantial memory and computational demands. Traditional federated learning (FL) approaches struggle with heterogeneous devices, and even parameter-efficient fine-tuning (PEFT) methods like LoRA can still require significant memory for intermediate activations. Split Learning (SL) and Split Federated Learning (SFL) offer solutions by dividing the model between clients and servers, but existing SFL methods often assume homogeneous client models or require the server to maintain multiple copies of server-side models, leading to scalability issues and high server memory/computation burdens. This paper addresses the need for an SFL framework that can handle heterogeneous devices efficiently, minimizing memory pressure on both clients and the server.
Methods
The proposed solution is an "edge-assisted split federated learning framework" that leverages LoRA for parameter-efficient fine-tuning. The core idea is to split the LLM into client-side and server-side submodels.
- Heterogeneous Client-Side Adaptation: Mobile devices, with varying capacities, perform LoRA fine-tuning on a subset of the LLM's lower layers. This "client-side submodel" is tailored to each device's capabilities.
- Server-Side LLM Reuse: The server maintains a full pre-trained LLM. Instead of replicating server-side models for each client, it selectively fine-tunes LoRA modules for each device sequentially. This reuse of the entire LLM significantly reduces server memory requirements.
- Parallel-Sequential Fine-Tuning: The training process is divided into two phases:
- Parallel Client-Side Training: Clients perform forward propagation on their assigned submodels.
- Sequential Server-Side Training: Activations from clients are sent to the server, which then performs forward and backward propagation on its LLM and updates the corresponding LoRA adapters. This sequential processing on the server avoids parallel computation bottlenecks and memory fragmentation.
- LoRA Adapter Aggregation: Periodically, LoRA adapters from clients are aggregated on the server to update both server-side and client-side LoRA adapters.
- Training Order Scheduling: A crucial component is a scheduling algorithm that determines the order in which clients' server-side training is processed. This algorithm prioritizes clients based on a combination of their LoRA adapter count and computing capacity to minimize overall training time by hiding communication and client computation under server computation.
Datasets
The paper uses the CARER dataset for text analysis tasks. The specific details of how this dataset is split or preprocessed for the experiments are not explicitly detailed beyond its use for fine-tuning a BERT-base model.
Key Insights and Ideas
- LLM Reuse is Key to Server Efficiency: The most significant insight is that by reusing the entire pre-trained LLM on the server for sequential LoRA updates, rather than maintaining multiple server-side submodels, memory and computational pressure on the server can be drastically reduced.
- Heterogeneity is an Opportunity, Not Just a Challenge: The framework explicitly accounts for device heterogeneity by allowing clients to adapt only a portion of the LLM based on their capacity, making fine-tuning feasible even on low-end devices.
- Strategic Scheduling Unlocks Performance: The proposed training order scheduling algorithm is critical for minimizing overall training time. By intelligently ordering client processing, the framework can effectively hide communication latency and client computation within the server's processing pipeline.
- SFL for LLMs is Viable on Edge: The work demonstrates that SFL can be effectively applied to LLM fine-tuning in resource-constrained edge environments, overcoming limitations of previous SFL approaches.
Key Results
The proposed memory-efficient SFL framework demonstrates significant improvements:
- Memory Reduction: Achieves a 79% reduction in memory footprint on the server compared to SFL.
- Training Time Reduction: Reduces training time by 6% compared to SFL and 40% compared to Split Learning (SL).
- Comparable Performance: Maintains accuracy and F1 scores comparable to baseline methods.
- Fastest Convergence: Achieves the fastest convergence time among the compared schemes.
- Scheduling Benefits: The proposed scheduling method shows improvements over FIFO and Workload-First scheduling in terms of training time.
Limitations and Future Work
- Limitations:
- The paper focuses on a specific LLM architecture (BERT-base) and a particular PEFT method (LoRA).
- The performance gains are measured on a simulated heterogeneous environment with specific device configurations.
- The impact of Non-IID data on the proposed scheme is mentioned but not extensively analyzed in terms of its specific effect on the framework's performance compared to baselines.
- Future Work:
- The authors suggest exploring more advanced scheduling algorithms to further optimize training time.
- Investigating the framework's applicability to other LLM architectures and PEFT techniques would be a natural extension.
- Further analysis of the impact of data heterogeneity and communication reliability on the framework's robustness could be beneficial.
Summary updated