Backdoor Persistence and Forgetting in Federated Learning

12 concepts10 papers1 proposal

Federated Learning (FL) is designed to act as a massive decentralized averaging machine. By aggregating updates from thousands of independent clients, the central global model learns generalized features without directly accessing raw user data. However, this distributed architecture introduces a unique security vulnerability: backdoor attacks, where a malicious participant injects poisoned updates to make the model misbehave on specific triggered inputs.

Yet, attackers face a fundamental mathematical hurdle. Once a malicious client contributes a poisoned update and rotates out of the training pool, the subsequent stream of updates from honest clients continues to optimize the model on clean data. This process triggers a phenomenon akin to catastrophic forgetting. The global model naturally "washes away" or dilutes the malicious parameter changes over successive rounds of training, causing the backdoor to rapidly fade before the model is ever deployed.

Understanding why backdoors decay—and how adversarial strategies attempt to make them "sticky"—has become a central focus of federated learning security research.


The Mechanics of Parameter Erasure

To understand why backdoors are inherently fragile, we must look at how updates are aggregated and how neural networks optimize. When an adversary modifies a set of weights to embed a backdoor trigger, those changes are immediately averaged with the updates of honest clients participating in the same round. As noted by Ashwinee Panda et al. in their work on Neurotoxin, the sheer volume of honest contributions dilutes the malicious payload right from the start.

Furthermore, the timeline of the training cycle dictates the survival rate of an attack. Optimization is not uniform across time:

  • Early-Stage Volatility: Early in training, the global model is far from convergence. Gradients are large, and honest clients are making massive trajectory shifts to learn core features.
  • The Dilution Effect: If an attacker injects a backdoor during these early rounds, the subsequent massive updates from benign clients completely overwrite the malicious parameter coordinates. As demonstrated by Tian Liu et al. in Assisting Backdoor Federated Learning with Whole Population Knowledge Alignment in Mobile Edge Computing, single-shot attacks executed too early face severe degradation because they cannot withstand this high-variance optimization phase.

As a result, standard backdoor attacks suffer from a lack of durability (Thuy Dung Nguyen et al.; Yanbo Dai et al.). Without active intervention, the network's natural learning process treats adversarial modifications as noise and optimizes them out of existence.


Engineering Durability: How Attackers Fight Forgetting

To bypass this natural defense, recent research explores how attackers can craft updates that resist being overwritten. These strategies generally fall into four intuitive mechanisms: exploiting parameter inertia, look-ahead optimization, manipulating global convergence, and representation anchoring.

1. Exploiting Parameter Inertia

Not all parameters in a neural network are updated equally by honest users. In any given training round, certain weights receive significant gradient updates, while others remain relatively stagnant.

Ashwinee Panda et al. exploited this insight by designing an attack that calculates which parameter coordinates are rarely updated by benign clients. By confining the backdoor modifications strictly to these inactive coordinates, the attack ensures that subsequent honest updates leave the malicious changes untouched. Building on a similar principle of targeting dormant spaces, Nader Bouacida et al. proposed layerwise model poisoning in Persistent and Stealthy Backdoor Attacks in Federated Learning via Layerwise Model Poisoning, focusing updates on specific, less-frequently updated layers to maximize backdoor lifespan.

2. Look-Ahead Optimization

Standard attacks optimize purely for the current global model state. However, an attacker can treat the FL pipeline as a game of chess, anticipating how the central server will average out updates in the future.

In Thinking Two Moves Ahead: Anticipating Other Users Improves Backdoor Attacks in Federated Learning, Yuxin Wen et al. introduced a look-ahead optimization framework. During local training, the attacker simulates future optimization rounds of honest clients. By calculating how honest clients are likely to alter the weight landscape, the attacker can inject a payload that aligns with or resists those future trajectories, effectively neutralizing the "washing away" effect before it happens.

3. Manipulating Global Convergence

If large updates from honest clients destroy backdoors, an attacker can try to shrink the step sizes of those honest clients.

To make early-stage attacks viable, Tian Liu et al. designed a two-phase attack strategy in Facilitating Early-Stage Backdoor Attacks in Federated Learning With Whole Population Distribution Inference. Instead of attacking immediately, the malicious client initially acts as an ideal, highly cooperative participant. By inferring the global population distribution, it uploads updates that artificially accelerate the global model's convergence. Once the model stabilizes early, the honest clients' gradient step sizes naturally shrink. The attacker then injects the backdoor, which safely persists because the benign updates are no longer large enough to overwrite it.

4. Representation Anchoring

Instead of just changing raw weight values, attackers can alter how the model structures its internal representations.

Yanbo Dai et al. developed Chameleon (Chameleon: Adapting to Peer Images for Planting Durable Backdoors in Federated Learning), which uses contrastive learning to anchor backdoor triggers deep within the model's embedding space. By binding the trigger features tightly to the underlying data representations, the backdoor becomes resistant to geometric shifts in the decision boundary. Similarly, Tao Liu 0038 et al. proposed using combinatorial trigger patterns in Beyond Traditional Threats: A Persistent Backdoor Attack on Federated Learning, distributing the adversarial concept across multiple features so that erasing one part of the trigger does not disable the attack entirely.


Exploiting Volatility for Defense

The fact that backdoors naturally decay provides defenders with a powerful lever. If backdoors are inherently prone to catastrophic forgetting, defenses can be designed to actively accelerate this process or prevent the long-term accumulation of malicious updates.

[Malicious Update] ───► [Imperfect Filter] ───► [Minor Leakage] ───┐
                                                                   ▼
[Continuous Training Rounds] ───────────────────────────► [Cumulative Backdoor]
                                                          (Embedded over time)

From a client-side perspective, Jingwei Sun et al. introduced FL-WBC (White Blood Cell) (FL-WBC: Enhancing Robustness against Model Poisoning Attacks in Federated Learning from a Client Perspective). Recognizing that persistent backdoors rely on hiding within specific, un-updated parameter spaces, this defense directs benign clients to actively perturb those exact spaces. By introducing controlled noise into the coordinates where attacks typically hide, honest clients force the model to "forget" the sticky backdoor updates without degrading overall task accuracy.

However, continuous training environments introduce a secondary threat: backdoor leakage. As analyzed by Shuaiqi Wang et al. in Towards a Defense Against Federated Backdoor Attacks Under Continuous Training, even if a central server deploys robust filters, those filters are rarely perfect. In every round, a tiny fraction of a malicious update may bypass detection. Over hundreds of rounds of continuous training, these minor leakages accumulate, gradually embedding the backdoor into the model's core architecture.

To combat this long-term accumulation, the authors proposed Shadow Learning, a framework that trains a parallel "shadow" model alongside the main backbone. By applying strict filtering and strategic early stopping to the shadow model, defenders can isolate and flush out the slow accumulation of leaked updates, ensuring that the natural forgetting mechanisms of the network are reinforced over long horizons.

Key Foundational Work

Neurotoxin: Durable Backdoors in Federated Learning (2022) — contributed 4 concept elements to this direction.
Assisting Backdoor Federated Learning with Whole Population Knowledge Alignment in Mobile Edge Computing (2022) — contributed 3 concept elements to this direction.
Thinking Two Moves Ahead: Anticipating Other Users Improves Backdoor Attacks in Federated Learning (2022) — contributed 1 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_catastrophic_forgetting_in_federated_lea_cefed380,
  title = {catastrophic forgetting in federated learning},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/catastrophic-forgetting-in-federated-learning/rdr_cefed3807f4a495ca2d546e560da4f4c},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-19},
}

This exact version (version 1, frozen text):

@misc{lacuna_catastrophic_forgetting_in_federated_lea_cefed380_v1,
  title = {catastrophic forgetting in federated learning},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/catastrophic-forgetting-in-federated-learning/rdr_cefed3807f4a495ca2d546e560da4f4c/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-19},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake