Federated learning trains one model across many devices or institutions that keep their data private, and it is unusually sensitive to settings such as the client learning rate, the number of local steps, and the server's update rule. Re-running the whole federated job for each candidate setting is too expensive. The current answer is to tune while training: keep a small population of configurations, each with its own model, and periodically replace the worst with perturbed copies of the best. FedPop does this with random mutations, and its perturbation scale is one more knob nobody tunes for you. This note proposes replacing those mutations with quantum-behaved particle swarm optimization (QPSO). QPSO is a classical, gradient-free search method. It drops the velocities of ordinary particle swarms and instead samples each candidate's next position from a heavy-tailed distribution centred on an "attractor" built from good past solutions. The width of that distribution is set by how spread out the population currently is. The population therefore explores widely early and narrows automatically as it agrees, and the swarm learns which direction is good instead of only re-sampling around winners.
The obvious bridge between these two fields would be to swap classical PSO for QPSO in existing swarm-based federated methods, either in local weight training or in server aggregation. We argue against that bridge. It asks a population method to search millions of network weights, and QPSO's own literature lists scaling to such parameter spaces as unsolved. Hyperparameter search is different. It has roughly 5 to 10 dimensions, evaluations are noisy and expensive, and gradients do not exist. That matches the regime where QPSO has been validated (30-dimensional benchmark functions), and it is where a federated system pays most for bad search.
Core Claim
Our claim is that a population-based federated tuner whose moves are drawn from QPSO attractor distributions will reach equal or better accuracy than FedPop under the same communication budget, without a hand-set mutation scale. We further claim it degrades more gracefully when some clients report corrupted validation scores. The second claim rests on a specific design choice. We use the rank-weighted attractor selection of QPSO-RO and score each configuration by the median over the clients that evaluate it. As a result, one inflated score can at most move a configuration up the ranking; it cannot set the search direction directly.
The method draws on two QPSO papers. Convergence analysis and improvements of quantum-behaved particle swarm optimization proves global convergence for QPSO via contraction mappings and introduces the ranking-operator variant QPSO-RO, which ranked best on CEC2005. Quantum-behaved particle swarm optimization with Gaussian distributed local attractor point proposes GAQPSO, which jitters each attractor with a Gaussian whose width is the distance between the swarm's mean best position and the particle's own best. GAQPSO took total rank 36 against 42 for standard QPSO and 87 to 127 for PSO variants, and it beat standard QPSO on 9 of 12 functions. The same paper also constrains our design: GAQPSO converges slowly late in the search on Griewank, and it lost to plain QPSO on unimodal functions. Our setting needs fine exploitation near the end of training, so we treat late-stage spread as a risk to measure, not as a free benefit.
Research Program
Each particle $i$ is a hyperparameter vector $x_i$ paired with its own copy of the global model. The vector covers log client learning rate, local epochs, local batch size, weight decay, and server learning rate and momentum, close to FedPop's split into server-side and client-side settings. Training runs in windows of $K$ communication rounds. During a window each particle's model trains under its own configuration on its share of sampled clients. At the end of a window each particle is scored by the median validation loss reported by a few clients. Each report is a single number, which keeps the uplink cheap in the spirit of the score-based messages in FedPSO.
The swarm keeps a personal best $P_i$ for each particle, a ranking of those personal bests, and their mean $C$. Particle $i$ forms an attractor $p_i = \varphi P_i + (1-\varphi) P_{r(i)}$. Here $\varphi$ is uniform on $[0,1]$, and $P_{r(i)}$ is a partner chosen at random with probability increasing in its rank, following QPSO-RO. In the GAQPSO variant the attractor is replaced by a draw from $\mathcal{N}(p_i, |C - P_i|)$. The new configuration is then
$$x_i' = p_i \pm \beta\,|C - x_i|\,\ln(1/u),$$
where $u$ is uniform on $(0,1)$, the sign is chosen at random, and $\beta$ is the contraction-expansion coefficient. This is the standard QPSO sampling rule. Its log-uniform term gives the heavy, infinite-support tails, and $|C - x_i|$ shrinks automatically as the swarm agrees.
Federated tuning differs from benchmark optimization in two ways, and both need new mechanisms. We add these because both directly decide whether the method works.
The first is weight inheritance. A particle's configuration is coupled to a partially trained model, so when a particle moves toward its partner it also copies the partner's model weights if the partner ranks higher. This is the "exploit" step of population training, with copy targets chosen by the attractor rather than by truncation.
The second is stale personal bests. A validation loss recorded at round 50 cannot be compared with one recorded at round 500, because training itself lowers loss. We compare two fixes. One expires each personal best after a fixed number of windows. The other defines fitness as the loss reduction achieved within the current window, which stays far more comparable over time.
The analysis component tracks swarm spread $|C - x_i|$ against rounds. This tests whether the automatic narrowing happens on federated schedules or whether the Griewank-style late slowdown reappears. If it does reappear, we test a decay on $\beta$ as the fix.
Baselines
The naive reference point is FedAvg with a single configuration chosen by a small grid on a held-out pilot. The pilot's cost is charged to the method's budget. Every tuner must beat this at an equal total number of client-rounds, or tuning is not worth it.
The strongest existing competitor is FedPop. It tunes while training with random perturbations and replacement of the worst configurations. It is the closest method in mechanism and claims the best results among tune-while-training approaches. We are aware of no published number for FedPop's absolute accuracy. The paper's headline result is an accuracy gain of up to 4.79% over FedEx on heterogeneous CIFAR-10 with a Successive Halving wrapper, and no reported result was found on Lacuna for its absolute CIFAR-10, FEMNIST, or Shakespeare accuracy. We therefore re-run FedPop ourselves. It gets the same population size, the same per-particle client allotment, the same evaluation windows, and the same search space, and its mutation scale is tuned on a pilot task with the same budget we give our $\beta$. Beating it means higher mean final test accuracy across at least five seeds at matched client-rounds, with non-overlapping confidence intervals, on at least two of three datasets. Matching its accuracy with no tuned perturbation scale would also count, as a weaker win.
For black-box tuning, FedHPO-Bench finds Bayesian optimization with Gaussian processes (BOGP) consistently strong. That benchmark report gives no numerical scores, so no reported result was found on Lacuna for BOGP accuracy on these tasks. We run BOGP and FedEx through the benchmark's own implementations.
The key ablations isolate what QPSO adds. Classical PSO over the same hyperparameters tests whether velocity-free sampling matters. QPSO with uniform instead of rank-weighted partner selection isolates the ranking operator's noise robustness. GAQPSO versus plain QPSO isolates the Gaussian attractor.
Why not just…?
Why not just give FedPop a decaying or spread-scaled mutation size? This is the real threat. If QPSO's benefit is only that its step size shrinks with population spread, then FedPop with perturbations scaled to $|C - x_i|$ captures it with no swarm machinery. We run exactly that variant. If it matches QPSO, the honest conclusion is that adaptive scale is the whole story and the attractor structure adds nothing. We would report that result.
Why not just use Bayesian optimization, which FedHPO-Bench found strong? BOGP treats each configuration as a full training run, and FedHPO-Bench itself found that methods fusing tuning into training generalize better than external wrappers. It also shows that a few noisy clients can mislead the server's hyperparameter choice, which is the failure our median-and-rank design targets. We still run BOGP at matched cost, in FedHPO-Bench's surrogate mode, as a sanity check of QPSO purely as a black-box optimizer.
Why not just apply QPSO to the model weights, extending Evolutionary Federated Learning Using Particle Swarm Optimization? That paper replaced local gradient descent with classical PSO and reported 0.9624 against 0.6814 accuracy on MNIST. A gradient-descent accuracy of 0.68 on MNIST strongly suggests an undertuned baseline rather than a property of swarms. FedPSO likewise saw only about a 0.1% difference from FedAvg on MNIST. We include one small check: QPSO over the weights of the two-layer FedPSO CNN against tuned SGD. We expect this to confirm that weight-space swarms do not compete, which is why we put QPSO in hyperparameter space instead.
Proposed Experiments
The main experiments use FedHPO-Bench's raw mode on FederatedScope. We run CIFAR-10 with a Dirichlet label split across 100 clients and a ResNet-sized network, FEMNIST split by writer, and Shakespeare split by speaking role. Population size is 8 to 16, evaluation windows are 10 to 20 rounds, and the total client-round budget is fixed per dataset across all methods. Metrics are final test accuracy, accuracy against client-rounds (area under the curve), server memory, uplink bytes for scores, and variance across seeds.
The decisive checks are three. First, QPSO-RO must beat FedPop at matched budget, and the spread-scaled FedPop ablation must not close the gap. Second, under FedHPO-Bench's noisy-client setting, with 0%, 5%, 10%, and 20% of evaluators returning random scores, the accuracy loss for the rank-weighted swarm must grow more slowly than FedPop's and BOGP's. Third, the curves of swarm spread against rounds must show whether late-stage spread costs final accuracy, with and without $\beta$ decay. Results on the windowed-improvement versus expiring personal-best fitness will tell whether stale scores, not search strategy, dominate. That would be a useful negative finding for population tuning in general.
References
- Convergence analysis and improvements of quantum-behaved particle swarm optimization
- Quantum-behaved particle swarm optimization with Gaussian distributed local attractor point
- FedPSO: Federated Learning Using Particle Swarm Optimization to Reduce Communication Costs
- FedPop: Federated Population-based Hyperparameter Tuning
- FedHPO-Bench: A Benchmark Suite for Federated Hyperparameter Optimization
- Evolutionary Federated Learning Using Particle Swarm Optimization