Standard federated learning relies on an iterative dance. In algorithms like Federated Averaging (FedAvg), clients download a global model, train it for a few local steps, upload their updated weights or gradients, and repeat this back-and-forth process hundreds or thousands of times. This iterative synchronization creates massive communication bottlenecks, requires clients to maintain persistent high-bandwidth connections, and repeatedly exposes intermediate model states to potential privacy leaks.
One-shot federated learning completely re-imagines this workflow by limiting the entire process to a single round of communication. Clients train their local models to completion or extract rich statistical summaries entirely independently, sending their information to a central server exactly once.
Surprisingly, this restriction does not necessarily doom model performance. Chor et al. demonstrated both theoretically and empirically that more frequent communication does not automatically translate to smaller generalization errors. In fact, averaging local models just once at the end of training can achieve the theoretical minimum for generalization error, avoiding the overfitting and training oscillations that continuous synchronization can introduce.
Why One Round Can Be Enough
To understand why a single round of aggregation can succeed, we must look at the loss landscapes of the models being trained.
For modern large-scale foundation models, one-shot aggregation is remarkably effective. As Wang et al. observed, because foundation models are pre-trained on massive datasets, they begin the federated fine-tuning process within a highly optimized, smooth loss landscape. This shared initialization acts as a powerful anchor: even when clients fine-tune their models on highly distinct local datasets, their optimization paths do not diverge drastically. A single weight-averaging step at the end of local training is sufficient to merge their insights without destroying performance.
Iterative Federated Learning:
Client 1: [Init] ---> [W1] ---> [W2] ---> [W3] ... ---> [Final]
| ^ ^ ^
Server: v | | |
[Avg] -------+--------+--------+
Client 2: [Init] ---> [W1'] ---> [W2'] ---> [W3'] ... ---> [Final']
One-Shot Federated Learning:
Client 1: [Init] -----------------------------------------> [Final] ---\
v
Server: [Aggregation]
^
Client 2: [Init] -----------------------------------------> [Final'] --/
However, when training standard neural networks from scratch, simply averaging the weights of independently converged models fails. Because neural networks are permutation-invariant, two networks trained on different data can learn identical features but assign them to entirely different hidden neurons. Directly averaging their raw weights results in "coordinate conflicts" that completely dismantle the learned representations.
To make one-shot learning work across diverse architectures and data distributions, researchers rely on three core mechanisms: knowledge distillation, structural alignment, and summary statistic reconstruction.
Mechanism 1: Knowledge Distillation and Generative Proxies
Instead of forcing divergent model weights to align, we can aggregate what the models know. In this paradigm, clients train their local models to completion and send them to the server. The server then uses an independent dataset or a generative model as a medium to transfer knowledge from the client models into a single global model.
Guha et al. pioneered this by showing that fully trained local models can be ensembled or distilled using unlabeled proxy data on the server, capturing enough structural information to reach roughly 90% of centralized performance in a single round. To capture deeper structural reasoning during this transfer, Gong et al. introduced FedAD, which distills both raw predictions (logits) and structural reasoning (attention maps) using unlabeled public data.
When auxiliary public data is unavailable, server-side generative models can synthesize the necessary proxy data. Sun et al. developed FedDiff, which leverages the high-fidelity generation of Diffusion Models combined with Fourier Magnitude Filtering to safely reconstruct data distributions under strict differential privacy constraints. Similarly, Gu et al. used latent diffusion models to disentangle client data distributions into distinct sub-tasks, bypassing weight-averaging bottlenecks entirely by allowing the server to train a global model on the synthesized global distribution.
Mechanism 2: Structural Alignment and Parameter Matching
A second strategy fixes the permutation problem directly by un-shuffling the hidden units of client models before averaging them.
Yurochkin et al. addressed this with Probabilistic Federated Neural Matching (PFNM). PFNM uses Bayesian nonparametrics to evaluate the activation patterns of neurons across different client networks, matching and aligning neurons that fire on similar features. By sorting the parameters into a shared structural order before aggregation, the server can safely average the weights without destroying the underlying network functions.
Permutation Problem:
Client 1 Neuron Order: [Detects Edges] [Detects Textures]
Client 2 Neuron Order: [Detects Textures] [Detects Edges]
Naive Average: [Mismatched] [Mismatched]
Structural Alignment (PFNM):
Client 2 Reordered: [Detects Edges] [Detects Textures]
Aligned Average: [Refined Edges] [Refined Textures]
Alternatively, the server can look for a shared geometric space where all client models remain optimal. Su et al. introduced a method called MA-Echo, which utilizes null space projection. By identifying the parameter subspaces that do not alter each client's local performance, the server can calculate a single global parameter set that satisfies all local constraints simultaneously, eliminating the need for server-side training or proxy datasets.
Mechanism 3: Exact Reconstruction via Summary Statistics
For statistical models like regressions, causal inference, and clustering, one-shot learning can be mathematically lossless—meaning it achieves the exact same precision as pooling all raw data into a single centralized database. This is possible because these models depend on a finite set of sufficient statistics.
In multi-site clinical research, maintaining persistent connections is often impossible due to administrative barriers. Luo et al. demonstrated that for linear mixed models, if local sites compute and transmit specific summary statistics—such as cross-product matrices and sample sizes—the central server has mathematically sufficient information to reconstruct the exact global model parameters in a single step. For logistic regressions, Duan et al. showed that sites can calculate local gradients based on an initial rough estimate from a lead site, providing a one-shot correction vector that yields near-perfect centralized accuracy. This principle of single-round summary statistic exchange has been successfully extended to survival risk models (Wang et al.) and distributed causal inference (Han et al.).
When optimizing more complex loss landscapes, sending a single point estimate (like a local minimum) strips away crucial context. Sharif-Nassab et al. solved this by having nodes communicate information about their local loss landscapes rather than just their best-fit parameters. Their Multi-Resolution Estimator (MRE) allows the server to map out the global loss surface and find the true global minimum in one shot, even when individual nodes hold very little data. Similarly, Wang utilized debiased local estimates combined with group hard thresholding to identify globally relevant features for multi-task learning in a single communication round.
This structural reconstruction strategy also applies to unsupervised learning. In federated clustering, Xu et al. demonstrated that global $k$-means cluster centers can be recovered in one round by treating local client centroids as pieces of a jigsaw puzzle and filtering out spurious local artifacts. To guarantee privacy and handle extreme data heterogeneity, Yan et al. used secure coded computing to reconstruct the global pairwise distance matrix directly in a single round, bypassing the accuracy degradation that occurs when averaging biased local cluster centroids.
Non-Parametric and Retrieval-Based Alternatives
A final class of one-shot methods avoids aggregating model parameters altogether, shifting the focus to exchanging representations or memorized data points.
Instead of synchronizing weights, Zhang et al. proposed exchanging class-representative embeddings, or prototypes. By centralizing these compact representations from all participants, a global adapter can be trained to convergence on the server in a single round, completely avoiding the convergence delays of weight synchronization.
Taking this concept to its logical conclusion, Du et al. introduced FedNN for neural machine translation. Rather than participating in gradient-based training, clients generate and upload localized $k$-nearest neighbor ($k$NN) datastores in a single round. The central model then performs retrieval-based inference against this global datastore. By shifting from model-based synchronization to memorization-based interaction, the system achieves instant domain adaptation with a fraction of the computational overhead.