TL;DR
FedE2E is a novel "one-shot" federated learning framework that builds a high-performance autonomous driving model by distilling knowledge from multiple pre-trained client models in a single communication round. By aligning disparate feature spaces and using a token-wise consensus mechanism, it achieves state-of-the-art closed-loop driving performance while drastically reducing the communication overhead typical of collaborative training.
What Problem Does This Paper Address?
End-to-end (E2E) autonomous driving—where a neural network maps raw sensor data directly to control commands—requires massive amounts of diverse data. However, privacy concerns and data regulations often prevent organizations from sharing their local datasets.
Federated Learning (FL) offers a solution by allowing models to be trained collaboratively without moving raw data. But standard FL has two major drawbacks for real-world driving:
- Communication Overhead: It usually requires hundreds of rounds of back-and-forth communication between a central server and clients, which is bandwidth-intensive and impractical if clients can only share their models once.
- Evaluation Gap: Most FL research relies on "open-loop" metrics (like how well a model predicts a recorded trajectory). In reality, driving is "closed-loop"—errors accumulate over time as the car interacts with the environment. A model that looks good on paper might crash in a simulator.
Furthermore, models trained in different cities (Non-IID data) develop different internal "languages" (feature representations), making them difficult to merge effectively.
Methods
The authors propose FedE2E, a framework that treats uploaded client models as "teachers" to train a single "student" (global) model on the server. The process happens in three main stages:
- Multi-client Calibration (MCC): Client models often have Batch Normalization (BN) layers tuned to their specific local cities. FedE2E uses a small, unlabeled server-side dataset to "re-calibrate" these BN statistics, ensuring all teacher models are speaking the same statistical language before distillation.
- Anchor-based Alignment (ABA): Even with calibration, different models might represent the same concept (e.g., a "pedestrian") using different numerical patterns. ABA uses the initial student model as an "anchor" and applies an orthogonal transformation (via Singular Value Decomposition) to map all teacher features into a shared representation space.
- Token-wise Consensus Distillation (TCD): Instead of just averaging the teachers' outputs, FedE2E looks at individual "tokens" (parts of the feature map). If most teachers agree on a specific feature, it's given a higher weight. This "consensus" approach ensures the student learns the most reliable, shared knowledge across the fleet.
Datasets
The researchers evaluated FedE2E using the Bench2Drive benchmark within the CARLA 0.9.15 simulator.
- Training Data: They used four distinct maps (Town03, Town12, Town13, and Town15) to simulate a Non-IID federated setting where each client has data from only one town.
- Evaluation: The models were tested across 220 evaluation routes, focusing on complex scenarios like non-signalized junctions, pedestrian crossings, and highway exits.
Key Insights and Ideas
- One-Shot is Enough: The paper demonstrates that you don't need constant communication to build a good model. A single upload of weights, if handled with sophisticated distillation, can outperform traditional multi-round methods.
- The Power of Alignment: The ablation study showed that Anchor-based Alignment provided the single largest boost in performance (improving the Driving Score by over 5 points). This highlights that the "representation gap" is the biggest hurdle in merging heterogeneous models.
- Consensus over Averaging: Simple parameter averaging (like FedAvg) often leads to "feature wash-out" in one-shot settings. By weighting features based on teacher agreement, FedE2E preserves sharp, actionable driving knowledge.
Key Results
FedE2E sets a new bar for one-shot federated driving:
- Driving Score (DS): In the one-shot setting, it achieved a 55.85 DS, outperforming the previous best method (FedDF) by 4.07 points.
- Success Rate (SR): It reached a 24.7% success rate, a significant jump over the 20.6% of prior work.
- Multi-round Efficiency: When allowed five rounds of communication, FedE2E reached a 60.35 DS. Notably, it achieved a better score in just 3 rounds than other methods did in 5, representing a 40% reduction in required communication.
- Safety: It matched the Infraction Penalty (IP) of a centralized model (0.78), indicating it drives as safely as a model trained on all data combined.
Limitations and Future Work
While FedE2E is a major step forward, the authors note a few areas for growth:
- Architecture Scaling: The current work uses a ResNet34 backbone. Testing how these distillation techniques scale to larger Vision Transformers (ViTs) or more complex multi-modal architectures is a logical next step.
- Diverse Environments: While Bench2Drive is rigorous, testing on even more diverse, real-world-inspired maps would further validate the model's generalization capabilities.
- Server-side Data: The method relies on a small proxy dataset at the server for distillation. Exploring "data-free" distillation methods could make the framework even more flexible.
Summary updated