TL;DR
Sense4FL is a novel federated learning (FL) framework designed for autonomous driving that dynamically optimizes vehicle selection and data collection based on driving trajectories. By mathematically minimizing the statistical distance between locally harvested data and the true regional distribution, it significantly improves object detection accuracy and accelerates model adaptation across different urban environments.
What Problem Does This Paper Address?
While autonomous driving (AD) vision models have made massive strides, they still struggle to generalize under localized domain shifts, such as unique street layouts, regional vehicle types, or extreme weather conditions. To improve these models on the fly without compromising user privacy or overloading cellular networks with massive raw video uploads, researchers turn to Federated Learning (FL). In FL, vehicles train models locally on their onboard computers and only upload model weights to a central server.
However, existing vehicular FL schemes suffer from a major blind spot: they assume that vehicles have static, pre-determined datasets. In reality, autonomous vehicles continuously collect data as they drive. Because different routes (trajectories) expose vehicles to vastly different environments—some streets are packed with pedestrians, while others are dominated by trucks—ignoring a vehicle's path leads to highly biased local datasets. This statistical imbalance (non-IID data) severely degrades the performance of the aggregated global model.
Methods
The authors introduce Sense4FL, a framework that treats vehicles simultaneously as mobile crowdsensors and FL clients.
The core technical contributions include:
- Convergence Analysis via Earth Mover’s Distance (EMD): The authors derive a new convergence upper bound for FL that accounts for both uncertain vehicle trajectories and wireless upload probabilities. They prove that minimizing training loss is mathematically equivalent to minimizing a weighted sum of local and global EMD between the vehicles' collected data distributions and the target regional distribution.
- Problem Formulation: They formulate a joint vehicle selection and data collection problem aimed at minimizing this EMD-based loss. The problem is structured as a non-linear multiple-choice knapsack problem (MCKP), which is NP-hard due to its non-convex, non-separable objective function.
- Two-Step Approximation Algorithm: To solve this efficiently on the server side, they develop a low-complexity algorithm:
- Step 1 (Bisection): Solves a fractional programming problem using a bisection method to minimize client-level data divergence.
- Step 2 (Local Search): Refines the vehicle selection by iteratively swapping out the least effective vehicles to optimize global distribution alignment.
Datasets
The framework was evaluated using the nuImages dataset (an extension of the popular nuScenes dataset designed for 2D object detection).
- Training/Evaluation Locations: Data from Singapore (One-North) was used to train models from scratch, and data from Boston (Seaport) was used to evaluate cross-city model adaptation.
- Traffic Simulation: Real-world vehicle mobility and trajectories were simulated using the SUMO (Simulation of Urban MObility) simulator across 36 distinct geographic blocks.
Key Insights and Ideas
- Trajectories Dictate Data Quality: A random selection of vehicles might over-sample common objects (like cars) and completely miss rare but critical classes (like traffic cones or strollers). Selecting vehicles based on where they are going ensures a statistically representative dataset for the entire region.
- The EMD Breakthrough: Linking FL convergence bounds directly to Earth Mover's Distance provides a clean, geometric way to optimize client selection based on semantic data distributions rather than just raw network bandwidth or compute capacity.
- Controlled Data Collection: Instead of telling vehicles to collect data indefinitely, Sense4FL optimizes when a vehicle should stop collecting data and start training, balancing the trade-off between data diversity and the strict time limits of wireless communication windows.
Key Results
The authors compared Sense4FL against several standard benchmarks, including random selection, gradient-based selection, and uploading-centric selection.
- Higher Accuracy: Sense4FL consistently outperformed all benchmarks in mean Average Precision ([email protected] and [email protected]:.95). For instance, trajectory-aware selection yielded a 4.4% accuracy boost over trajectory-agnostic random selection in baseline tests.
- Superior Adaptation: When adapting a model pre-trained in Singapore to the new environment of Boston, Sense4FL converged much faster and achieved significantly higher precision and recall than other methods, proving its capability to handle domain shifts effectively.
- Robustness: The framework maintained its performance edge across varying network conditions, including low upload data rates and limited vehicle availability.
Limitations and Future Work
While Sense4FL provides a robust framework for object detection, the authors note a few areas for expansion:
- Task Scope: The current work focuses strictly on 2D object detection. Future iterations could extend the framework to other critical AD tasks like semantic segmentation or direct control command generation (e.g., acceleration/braking).
- Trajectory Complexity: The mobility model assumes relatively simple trajectory prediction parameters ($M_v = 2$ potential paths per vehicle). Incorporating more advanced, deep-learning-based intent and trajectory prediction models could improve selection accuracy in highly chaotic environments.
Summary updated