Batch Size as a Free Systems Knob in Federated Learning

Federated learning trains one model across many devices, such as phones, sensors, and embedded boards, without collecting their raw data. Each device trains locally for a while and sends back an update. Practitioners already know two painful facts about it. The first is that slow devices ("stragglers") hold up every round. The second is that the learning rate, weight decay, and anti-drift settings that work in one deployment often fail in another. Recent systems papers attack the first problem by giving each device a different batch size. Doing so quietly worsens the second problem, because a learning rate tuned for batch 32 is the wrong learning rate for batch 8 or batch 256. This proposal treats that coupling as the central issue. Real edge hardware makes small batches much more expensive per sample than simple latency models predict. We ask whether a rule that rescales the optimization hyperparameters with batch size lets a system pick each device's batch size purely for hardware speed, while keeping a single tuned configuration valid across the whole fleet.

The claim is concrete. First, per-step latency on edge accelerators is roughly flat in batch size up to a device-specific "knee," because small-batch training is limited by memory bandwidth rather than arithmetic. Batches below the knee therefore waste time without saving any. Second, once each device trains at its own knee, fixed hyperparameters become miscalibrated in a predictable way. Rescaling the local learning rate, weight decay, sharpness-aware perturbation radius, and local step count so that gradient noise and per-round parameter movement stay constant should remove most of that miscalibration. If both parts hold, batch size stops being a hyperparameter and becomes a systems setting.

Core Claim and Why It Is Not Already Done

Four closely related papers choose client batch sizes for latency. Optimal Batch-Size Control for Low-Latency Federated Learning With Device Heterogeneity equalizes per-round latency across devices with a closed-form batch rule. It reports 37.5% lower end-to-end latency on CIFAR-10, but it assumes plain averaging with no drift regularizer. DYNAMITE derives a square-root rule, $s_i \propto \sqrt{M_i D_i}$: the batch size $s_i$ of client $i$ grows with the square root of its gradient variance $M_i$ times its data size $D_i$. It jointly adapts aggregation frequency. ADEL-FL co-optimizes round deadlines and batch scaling under an exponential compute-time model. FedTune tunes participant count and local passes against weighted system costs.

These papers supply the motivation and the controllers we compare against. They share two assumptions we drop. They price computation as smooth in batch size, whereas the hardware literature shows it is not. They also hold the learning rate and regularizers fixed while batch size moves, whereas the federated optimization literature shows those settings are fragile.

The hardware evidence comes from several sources. EdgeDRNN and Streaming Architecture for Large-Scale Quantized Neural Networks on an FPGA-Based Dataflow Platform show that batch-size-one execution is bound by weight fetches from memory. MLPerf Inference Benchmark reports that latency-constrained serving loses over 50% of offline throughput.

The fragility evidence also comes from several sources. FedNAR reports up to 30% accuracy loss from a poorly chosen weight decay under heterogeneity. The sharpness-aware methods FedSMOO and FedNSAM depend on delicate settings of the perturbation radius $\rho$ and momentum. A Practical Recipe for Federated Learning Under Statistical Heterogeneity shows that increasing local epochs triggers client drift.

No paper in the file measures how hyperparameters tuned at one batch size behave after a latency controller changes it. No paper proposes making those hyperparameters batch-invariant so that the controller and the tuner can be separated.

Research Program

The work has three parts.

The first part is a measured cost model. For each device class, we profile one local training step (a forward and backward pass, doubled for sharpness-aware methods) across batch sizes from 1 to 512. We fit a two-regime curve: flat below a knee $b^\star$ and linear above it. We then check whether this curve, rather than the linear or exponential models in the prior controllers, predicts round time. The doubled pass matters in particular. Sharpness-aware methods cost roughly 1.5x more local compute, and below the knee that extra cost is paid mostly in memory traffic.

The second part is a rescaling rule. Let $\eta_0, \lambda_0, \rho_0, K_0$ be the learning rate, weight decay, perturbation radius, and local step count tuned once at a reference batch $b_0$. A client with batch $b$ uses the values below, where $\alpha\in\{1/2, 1\}$ is chosen per optimizer:

$$ \eta = \eta_0 \left(\tfrac{b}{b_0}\right)^{\alpha},\qquad K = K_0\,\tfrac{b_0}{b}\cdot\tfrac{\eta_0}{\eta}\cdot\left(\tfrac{b}{b_0}\right)^{0},\qquad \lambda\,\eta = \lambda_0\,\eta_0 . $$

The learning-rate exponent $\alpha$ is taken as 1 for SGD-style updates and 1/2 for adaptive ones. The step count $K$ is set so that each round's total displacement $\eta K$ matches the reference, which holds drift per round constant. Weight decay is held constant per unit of learning rate. For sharpness-aware methods, $\rho$ is scaled so that the perturbation is measured relative to the batch-gradient noise, which shrinks as $1/\sqrt{b}$. Server-side aggregation weights each client by the samples it processed, not by the steps it took. This avoids the inconsistency that arises when clients take different numbers of steps. None of these exponents is assumed correct. Each is an ablation arm, and the program's deliverable is which ones actually matter in the federated, non-IID regime.

The third part is a knee-aware controller. It sets each device's batch size to $\max(b^\star_i, s_i)$, where $s_i$ comes from a variance-based rule such as DYNAMITE's, and applies the rescaling rule automatically. Nothing is re-tuned when a device's batch changes.

Baselines

The naive reference is FedAvg with a fixed batch of 32 and hyperparameters tuned in a homogeneous simulation, then deployed unchanged on the heterogeneous fleet. We add local fine-tuning because the practical-recipe study found this simple combination often competitive.

The strongest systems competitors are ADEL-FL and the optimal batch-size controller, both run with the hyperparameters their papers use.

  • ADEL-FL reports 61% CIFAR-10 accuracy against FedAvg's 30% under a 1200-second wall-clock limit. Beating it means higher accuracy at the same wall-clock budget on the same simulated device mix.
  • The optimal batch-size controller reports 37.5% latency reduction on CIFAR-10 and 26.7% on MNIST. Beating it means lower time-to-target-accuracy with no per-deployment retuning.

The strongest variance-aware competitor is DYNAMITE. It reports 2.7% to 7.9% higher final accuracy and 45.4% to 63.8% faster time-to-target than no-straggler baselines. We run its online controller with and without our rescaling.

The strongest tuning competitor is FedEx from Federated Hyperparameter Tuning, which reports a 2-3% Shakespeare accuracy gain over random search and successive halving at equal budget. Here it retunes local hyperparameters after batch sizes are assigned.

Fairness rules apply to all comparisons. Every method sees the same Dirichlet partitions, the same measured device latency curves, and the same wall-clock budget. Tuning budgets are equal and counted in client-rounds; FedEx's search rounds are charged against its own budget.

Two ablations isolate the two ingredients. The first uses knee-aware batch selection without rescaling. The second uses rescaling with the prior controllers' smooth cost models. A third ablation removes one rescaled quantity at a time: learning rate only, then adding step count, then weight decay, then $\rho$. A method counts as successful if it matches FedEx's accuracy within one point at a fraction of the tuning cost, and beats ADEL-FL and DYNAMITE on time-to-accuracy.

Why not just…?

Why not just use the linear learning-rate scaling rule from centralized large-batch training? This is the most serious threat. If scaling the learning rate alone recovers most of the lost accuracy, the contribution shrinks to the cost model plus a known rule. The learning-rate-only ablation settles it directly. Two things make us expect more is needed. In federated training, raising the learning rate while keeping local steps fixed increases per-round drift, which the practical-recipe study identifies as the dominant failure mode. FedNAR also shows weight decay interacts with drift on its own. If learning-rate scaling suffices, we will report that as the result and redirect the contribution to the cost model and controller.

Why not just rerun FedEx whenever batch sizes change? FedEx can only tune client-side settings and cannot tune server momentum or the server learning rate. Its search also consumes client rounds. Under a moving batch assignment, it would have to search continually. We charge its search cost against the wall-clock budget and report whether it still wins at equal cost.

Why not just give every device its knee batch and keep the hyperparameters fixed? This is the first ablation. It is cheap and may capture most of the latency gain. However, it assigns different learning dynamics to fast and slow devices. If the fast devices' data is skewed, this could bias the global model, the same bias concern FedEL raises for dropped stragglers. The sensitivity measurements below will show whether accuracy degrades.

Why not just drop stragglers at a hard deadline? DYNAMITE and FedEL both argue that this discards non-IID information held by slow clients. We include it only as a reference line in the wall-clock plots.

Proposed Experiments

We use CIFAR-10 and CIFAR-100 with Dirichlet 0.1 and 0.3 splits, FEMNIST from LEAF, and Google Speech Commands. Training runs in FedML using latency curves profiled on at least three edge device classes. Local optimizers are FedAvg, FedNAR, and FedSMOO.

The decisive check is a transfer test. We tune once at batch 32 in a homogeneous simulation, then deploy on the heterogeneous knee-batch fleet. For each hyperparameter grid, we measure the accuracy gap and the width of the region of the grid within one point of the best result. The rescaling rule should keep this region at least as wide as in the homogeneous setting; without rescaling, it should narrow sharply. The expected sharpening for sharpness-aware methods follows from $\rho$'s dependence on gradient noise.

A second check validates the cost model. The two-regime curve should predict measured round time better than linear or exponential models, and the gap should be largest below the knee.

The final measure is time-to-accuracy against the baselines above, at equal tuning cost. If the rescaling offers no gain over learning-rate-only scaling, or if the knee is absent on real devices, the core claim fails and we will report which half broke.