Abstract
Fine-tuning large-scale foundation models (FMs) on uncrewed aerial vehicles (UAVs) is often hindered by the "straggler" effect and inefficient communication protocols. This paper introduces Communication-Pipelined Split Federated Learning (CPSFL), a novel framework designed to optimize training in resource-constrained UAV networks. By integrating sequential gradient transmission, priority-based scheduling, and asynchronous pipelining, CPSFL reduces training latency by over 30% compared to existing state-of-the-art methods like PipeSFL.
Introduction
The deployment of FMs, such as Swin Transformers, on UAVs facilitates advanced real-time tasks like facility inspection and environmental monitoring. However, the substantial memory and energy requirements of these models exceed the capabilities of typical UAV hardware. Split Federated Learning (SFL) mitigates this by partitioning the model: the UAV (client) processes initial layers, while a Base Station (BS) handles the remaining computation.
Despite its benefits, SFL faces a significant bottleneck: the straggler problem. In traditional SFL, training rounds progress only as fast as the slowest participant. Furthermore, parallel downlink transmissions often leave communication resources underutilized during server-side computation phases. In UAV networks, where communication latency typically outweighs computation time, these inefficiencies lead to substantial delays.
Proposed Method: CPSFL
To overcome these limitations, the authors propose CPSFL, which introduces three primary architectural innovations:
- Sequential Downlink (SFL-PS): Unlike traditional parallel transmission, the server allocates full downlink bandwidth to a single client at a time. This time-division approach optimizes the communication-heavy gradient transmission phase.
- Priority Scheduling: The framework employs a scheduling algorithm that prioritizes clients with the largest "lag"—the duration from the start of backward propagation to the completion of server-side compute. This allows slower clients to begin subsequent iterations earlier.
- Intra-round Asynchronous Training: The server initiates computation and gradient transmission for individual UAVs as soon as their data is received, rather than waiting for a global synchronization point. This creates a continuous "pipeline" of training activities.
Additionally, an Attention-based Deep Reinforcement Learning (DRL) agent at the BS manages resource allocation. By analyzing historical UAV trajectories, the agent predicts future channel conditions to dynamically adjust model split points and resource distribution (CPU frequency and bandwidth).
Experimental Evaluation
The framework was evaluated using the CIFAR-100 dataset to fine-tune a Swin-L model (~200M parameters) across a simulated 10-UAV network. UAV mobility was modeled using 3GPP RMa-AV standards.
Key Results
- Latency: CPSFL achieved a 30% reduction in per-round latency compared to the PipeSFL benchmark.
- Energy Efficiency: The DRL-driven resource allocation successfully balanced training speed with UAV battery constraints.
- Convergence: The attention-based DRL converged more rapidly and achieved higher rewards than standard DRL models, validating the importance of trajectory-aware optimization.
Conclusion
CPSFL addresses the critical communication and computation bottlenecks in UAV-based foundation model fine-tuning. By prioritizing "lagging" clients and pipelining the training process, the framework ensures efficient resource utilization in dynamic aerial environments. Future research will explore multi-base station handovers and dynamic LoRA rank adjustment to further enhance system scalability and performance.
Summary updated