Federated Learning of Car Aerodynamics Surrogates with Spectral Optimizers

Arman Bolatov, Nikolay Spitsyn, Daniil Medyakov, Vladimir Aletov, Dmitry Bylinkin, Martin Takáč, Aleksandr BeznosikovarXiv 2026 · arXiv

TL;DR

This paper demonstrates that spectral optimizers, which orthogonalize weight updates, are exceptionally effective for training federated neural surrogates for car aerodynamics. By introducing new methods like FedMARS-M, the authors achieved drag prediction accuracy within 2% of centralized models while keeping proprietary design data private across multiple manufacturers.

What Problem Does This Paper Address?

In the automotive industry, designing fuel-efficient or high-performance vehicles requires precise estimates of aerodynamic drag and lift. Traditionally, this is done via Computational Fluid Dynamics (CFD), which is notoriously slow—a single high-fidelity simulation can take hours on a massive computing cluster.

Neural surrogates offer a "fast-forward" button, predicting these forces in seconds. However, training them requires thousands of CFD simulations. Because these simulations represent proprietary car designs, manufacturers are unwilling to pool their data into a single centralized dataset.

Federated Learning (FL) offers a solution by allowing multiple "clients" (manufacturers) to train a shared model without ever exchanging their raw data. The challenge is that aerodynamic data is highly heterogeneous (different manufacturers make different body types) and the learning task is harder than standard image classification. Small errors in predicting surface pressure fields can lead to massive errors in the final integrated drag force. This paper investigates which federated optimizers can actually handle the rigors of physical simulation data.

Figure 1: Problem setting: (a) the airflow exerts a pressure and a wall shear stress on the surface of the car (colors are schematic), (b) integrated over the surface, the two fields give the drag and lift forces, and (c) each manufacturer trains on its own cars while only model updates are exchanged.

Methods

The authors utilize the Transolver architecture, a specialized transformer designed for large unstructured meshes. Unlike standard transformers that struggle with hundreds of thousands of points, Transolver groups points into "learned slices" to keep computation costs manageable.

The core contribution lies in the evaluation and proposal of spectral optimizers in a federated setting. While standard optimizers (like Adam or SGD) update parameters independently, spectral optimizers update each weight matrix as a whole by orthogonalizing the update.

The paper compares:

  • Spectral Optimizers: FedMuon and two novel proposals: FedMARS-M (which adds variance reduction to client momentum) and FedOLion (a federated version of the sign-based OLion).
  • Non-spectral Baselines: FedAvg, SCAFFOLD-Adam, and FedLion.

To combat "client drift"—where local models diverge because they see different types of cars—the authors adapt SCAFFOLD-style control variates to work with the non-linear orthogonalization steps of spectral optimizers.

Datasets

The study uses DrivAerNet++, a large-scale multimodal car dataset.

  • Scale: 5,703 training cars and 1,154 test cars, totaling ~125 GB of surface data. This is nearly three orders of magnitude larger than the MNIST or CIFAR benchmarks typically used in FL research.
  • Partitions: To simulate real-world heterogeneity, the data was split among six clients in four ways, ranging from "single-class" (one client only sees estatebacks, another only fastbacks) to nearly identical distributions using Dirichlet sampling.

Key Insights and Ideas

  • The "Orthogonalization Gain": The authors found that weight gradients in these models have a low stable rank. In this regime, an orthogonalized update (which treats all directions in the weight matrix equally) yields about twice the first-order decrease in loss compared to a standard gradient step of the same spectral norm.
  • Efficiency vs. Computation: Spectral optimizers are "computationally dense" in a good way. They perform best with only 2 to 4 local epochs per round. In contrast, non-spectral methods like FedAvg required 16 local epochs to get even close, yet still failed to match the accuracy of the spectral methods.
  • Stability in Heterogeneity: Spectral methods proved much more robust to data heterogeneity. Even when clients held entirely different car body types, FedMARS-M maintained high accuracy, whereas non-spectral methods saw their error rates climb significantly.

Key Results

  • Near-Centralized Accuracy: FedMARS-M and FedMuon reached a drag error within 2% of a model trained on centralized data.
  • Superiority over Baselines: The best non-spectral method (SCAFFOLD-Adam) still had a 17% higher drag error than FedMARS-M, while standard FedAvg was 37% worse.
  • Communication Efficiency: Because spectral methods require fewer local steps to reach high accuracy, they effectively reduce the total amount of computation needed on the client side to reach a target performance level.

Limitations and Future Work

  • Heuristic Explanation: The link between "low stable rank" and spectral optimizer performance is currently a heuristic explanation; a formal theoretical proof in the context of federated local steps is still missing.
  • Architecture Specificity: The results are based on the Transolver architecture. The authors plan to test if these insights hold for other architectures like OmniArch.
  • Communication Cost: While spectral methods are accurate, they currently require uploading full-precision updates and control variates, which doubles the communication volume per round compared to simple FedAvg. Future work will look into compressing these updates.
  • Real-world Noise: The study uses a single public dataset partitioned into clients. Testing with actual industrial partners whose data might differ in mesh resolution, solvers, and flow conditions is the ultimate next step.

Summary updated

Abstract

Neural surrogates predict the aerodynamic forces on a car in seconds, compared with hours for a CFD simulation, but training them requires thousands of CFD simulations of different car designs, which car manufacturers keep proprietary. Federated learning allows several manufacturers to train a shared surrogate without exchanging data. We study which federated optimizer brings such a surrogate closest to one trained on pooled data. On DrivAerNet++, we train a Transolver surrogate across six clients that each hold cars of a single body type, and we compare three spectral optimizers, which orthogonalize the update of each weight matrix (FedMuon and two methods we propose, FedMARS-M and FedOLion), with three non-spectral ones (FedAvg, SCAFFOLD-Adam and FedLion), all tuned with the same protocol. FedMuon and FedMARS-M reach a drag error within 2% of centralized training with Muon, using two to four local epochs per round. The non-spectral methods perform best with sixteen local epochs, the largest number we tested, and still have a 17 to 37% higher drag error than FedMARS-M. We relate this gap to the low stable rank of the weight gradients, for which an orthogonalized step yields about twice the first-order decrease of a gradient step of the same spectral norm. Code: https://github.com/armanbolatov/fedmars_drivaernet
BibTeX
@misc{bolatov2026federated,
  author = {Arman Bolatov and Nikolay Spitsyn and Daniil Medyakov and Vladimir Aletov and Dmitry Bylinkin and Martin Takáč and Aleksandr Beznosikov},
  title = {{Federated Learning of Car Aerodynamics Surrogates with Spectral Optimizers}},
  year = {2026},
  eprint = {2611.25300},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.25300},
}
Figure