Server-Proximal Aggregation for Federated Domain-Incremental Learning under Partial Participation: Task-Uniform Convergence and Backward Transfer

Longtao Xu, Jian Li

Real-world federated systems rarely operate on static data. Input distributions drift over time, yet privacy regulations strictly forbid the sharing of raw data. This creates a significant challenge for training robust machine learning models across distributed networks. To address this, researchers have introduced SPECIAL (Server-Proximal Efficient Continual Aggregation for Learning), a lightweight, replay-free algorithm designed to tackle Federated Domain-Incremental Learning (FDIL). By adding a single "anchor" step at the server level, SPECIAL prevents models from forgetting old tasks as they learn new ones, achieving state-of-the-art accuracy and theoretical convergence guarantees without extra data storage or complex architectures.

The Challenge of Federated Domain-Incremental Learning

Imagine a network of hospitals training a diagnostic AI. Over time, scanner hardware is upgraded, or patient demographics shift—a phenomenon known as domain shift. In federated learning (FL), where data remains on local devices for privacy, this introduces two primary hurdles: spatial heterogeneity (varying data across different hospitals) and temporal non-stationarity (data evolving over time).

The focus here is on Federated Domain-Incremental Learning (FDIL). Unlike class-incremental learning, where entirely new categories appear, FDIL deals with a fixed set of labels but shifting input distributions. The primary goal is to learn these new domains without suffering from "catastrophic forgetting" of previous ones.

Existing solutions often rely on experience replay (which risks privacy by storing old data) or parameter expansion (which inflates model size and reduces efficiency). Furthermore, existing theoretical frameworks rarely account for partial participation, where only a fraction of clients are available during any given training round.

The SPECIAL Approach: A Server-Side Anchor

To address these gaps, the authors propose SPECIAL. The algorithm builds upon the standard FedAvg framework by introducing a single, elegant modification: a server-side "anchor."

The process unfolds in three streamlined steps:

  1. Local Training: Participating clients perform standard Stochastic Gradient Descent (SGD) on their local, new-task data. They do not need to store historical data or compute complex local regularizers.
  2. Aggregation: The central server collects and averages these local updates.
  3. The Anchor Step: Instead of directly overwriting the global model, the server "blends" the newly averaged updates with the global model from the previous task using a lightweight proximal step.

This anchor acts as a compact memory. It prevents the model from drifting too far from its previously learned parameters, effectively balancing the stability-plasticity dilemma—retaining old knowledge while absorbing new information. Crucially, a major insight of this work is that regularizing at the server rather than the client allows local nodes to learn new domains freely without being bogged down by historical constraints.

Theoretical Guarantees: Convergence and Backward Transfer

SPECIAL provides two missing theoretical pillars for FDIL under partial participation:

  • Task-Uniform Convergence: The paper establishes a convergence rate of $\mathcal{O}\big(\sqrt{E/(NT)}\big)$ in expected gradient norm (where $E$ is local epochs, $N$ is participating clients, and $T$ is rounds). This matches the efficiency of standard FedAvg on stationary data while explicitly separating optimization variance from inter-task drift.
  • Backward Knowledge Transfer (BKT): The authors mathematically prove that training on new tasks can preserve or even improve performance on earlier tasks. The BKT bound caps any increase in earlier-task loss by a drift-controlled term that shrinks as rounds, local epochs, and client participation increase.

Experimental Validation

The researchers evaluated SPECIAL across diverse benchmarks, including vision datasets (Digit-10, VLCS, PACS, DN4IL) and an NLP benchmark (a sequence of GLUE tasks using DistilBERT).

  • Superior Accuracy: SPECIAL achieved the highest average accuracy across all tested datasets, outperforming both replay-free and replay-based baselines.
  • Efficient Convergence: It reached peak performance significantly faster. On the Digit-10 dataset, SPECIAL hit its best state in 63 rounds compared to 83 for standard FedAvg.
  • Zero Overhead: Because the anchor is a simple mathematical blend at the server, SPECIAL adds zero communication cost and requires no increase in model size.
  • Cross-Domain Success: The algorithm's strong performance on text classification tasks demonstrates that server-side anchoring is a general-purpose strategy applicable beyond computer vision.

Limitations and Future Directions

While highly effective, the current theoretical framework assumes synchronous participation and L-smooth loss functions, which may not perfectly reflect real-world systems dealing with "stragglers" (slow clients) or non-smooth objectives. Additionally, the proximal weight ($\lambda$) currently requires per-dataset tuning, paving the way for future research into self-tuning mechanisms.

Ultimately, SPECIAL demonstrates that mitigating catastrophic forgetting in federated networks does not strictly require complex memory buffers or massive architectures. Sometimes, a well-placed server-side anchor is all you need.

Summary updated

Abstract

Real-world federated systems seldom operate on static data: input distributions drift while privacy rules forbid raw data sharing. We study Federated Domain-Incremental Learning (FDIL), where (i) clients are heterogeneous, (ii) tasks arrive sequentially with shifting domains, and (iii) the label space remains fixed. Two theoretical pillars remain missing for FDIL under partial participation: a guarantee of backward knowledge transfer (BKT) and a convergence rate that holds *uniformly across the task sequence*. We introduce SPECIAL (Server-Proximal Efficient Continual Aggregation for Learning), a simple, replay-free FDIL algorithm that adds a single server-side ``anchor'' to FedAvg: in each round, the server aggregates updates from a uniformly sampled subset of clients and then blends the result with the previous global model via a lightweight proximal step. This anchor curbs cumulative drift without replay buffers, synthetic data, or task-specific heads, leaving communication cost and model size unchanged. Our theory shows that SPECIAL (i) *preserves earlier tasks*: a BKT bound caps any increase in earlier-task loss by a drift-controlled term that shrinks with more rounds, local epochs, and participating clients; and (ii) *achieves task-uniform, communication-efficient convergence* for non-convex FDIL with partial participation: $\mathcal{O}\big(\sqrt{E/(NT)}\big)$ in expected gradient norm, with $E$ local epochs, $T$ rounds, and $N$ participating clients, while explicitly separating optimization variance from inter-task drift. Experiments on standard FDIL benchmarks corroborate the theory.
BibTeX
@inproceedings{xu2026server,
  author = {Longtao Xu and Jian Li},
  title = {{Server-Proximal Aggregation for Federated Domain-Incremental Learning under Partial Participation: Task-Uniform Convergence and Backward Transfer}},
  booktitle = {International Conference on Machine Learning},
  year = {2026},
  url = {https://openreview.net/forum?id=UHuu4jgy2u},
}
Figure