Federated learning (FL) promises a powerful paradigm shift: training machine learning models on decentralized, privacy-protected data distributed across millions of edge devices. However, this privacy guarantee introduces a significant blind spot. Because a central server cannot directly inspect or verify raw user data, it cannot guarantee data quality.
In real-world deployments, decentralized data collection is notoriously messy. Non-expert annotations, sensor glitches, and varying institutional protocols inevitably introduce incorrect or "noisy" labels. When left unaddressed, this label noise poisons the collaborative training process, causing global models to degrade or completely diverge.
The Core Dilemma: Why Local Noise Destroys Global Models
To understand why label noise is uniquely destructive in federated environments, we must look at how neural networks learn from small datasets.
In a traditional, centralized training setting, a model exposed to random label noise can often bypass it. Because the dataset is massive, the true underlying statistical patterns outnumber the random errors, preventing the model from easily memorizing the noise.
Federated learning flips this dynamic. Individual edge clients typically possess very small, localized datasets. As Ji et al. observe in their work on FedFixer, these data-starved local models lack the diversity needed to average out errors. Instead, they rapidly overfit to and memorize the local noisy labels, warping their decision boundaries.
Standard Federated Learning with Noisy Labels:
[Clean Client] ----> Valid Gradients ---\
\---> [Central Server] ---> Aggregated Global Model
[Noisy Client] ----> Warped Gradients ---/ (Averages conflicting boundaries -> Degraded)
When the central server aggregates these local models (using standard algorithms like FedAvg), it blindly averages their weights. As Yang et al. demonstrate in Robust Federated Learning with Noisy Labels, combining these conflicting, noise-warped decision boundaries yields a degraded global model.
Compounding this issue is noise heterogeneity. Label noise is rarely uniform across a network; one client might have pristine data, while another contains 40% corrupted labels. As Yang et al. show in FedTrans, this imbalance acts as a severe bottleneck, accelerating client drift and drastically increasing the communication rounds required for the network to converge.
To solve this, researchers have developed three core strategies: leveraging global consensus, curating data at the edge, and learning noise-invariant features.
Strategy 1: Global Consensus as an Anchor
If local models overfit to noise due to a lack of data diversity, the global model represents the exact opposite: an aggregation of diverse insights that naturally filters out random, uncorrelated errors. Modern frameworks exploit this asymmetry by using the global model as a corrective signal for noisy local clients.
For instance, the authors of FLR introduce a label-mixture regularization technique. Instead of allowing a local client to train blindly on its own questionable labels, the system blends the global server’s insights with local predictions to generate robust pseudo-labels. This prevents the local model from entering the rapid noise-memorization phase.
Historically, many robust FL frameworks assumed that at least a few clients possessed perfectly clean data to act as a reference point. However, Jiang challenges this assumption with FedClean, proving that a global model can successfully orchestrate label correction even when every single participant's dataset is corrupted. By dynamically weighting client updates based on global consensus, the network can extract a clean signal from a universally noisy environment.
Strategy 2: Client and Sample Curation
Rather than attempting to fix corrupted labels during training, another intuitive approach is to filter them out entirely before they can poison the global model. This curation can happen at either the client level or the individual sample level.
Client Pruning
If a participant's data is overwhelmingly corrupted, their updates may do more harm than good. Morafah proposes an outright exclusion strategy called ClipFL. By calculating a Noise Candidacy Score (NCS) based on the behavior of incoming client updates, the server can identify highly corrupted "bad actors" and prune them from the federation entirely, accelerating convergence and saving communication bandwidth.
Sample-Level Filtering
Total exclusion can sometimes waste valuable, clean data trapped alongside noisy labels. To address this, Nagalapatti introduces Is Your Data Relevant?, a framework where clients use reinforcement learning to locally curate their own datasets, filtering out out-of-distribution or mislabeled samples before training occurs.
Similarly, Xu et al. design FedBSS, which isolates suspicious samples using loss and uncertainty metrics. Instead of discarding them, it progressively schedules their integration into the training process only after the model has stabilized on highly confident data points.
Sample Curation Pipeline (FedBSS):
[Raw Local Data] ---> [Loss/Uncertainty Filter] ---> High Confidence Samples ---> Immediate Training
---> Low Confidence Samples ---> Deferred/Scheduled
Strategy 3: Structural and Feature-Level Robustness
A third, more fundamental approach shifts the focus away from fixing labels or filtering data, altering what the network actually learns.
Standard optimization objectives focus heavily on mapping features to discrete, user-provided labels. If those labels are wrong, the gradients become toxic. To bypass this vulnerability, Zhong et al. propose FedCova, a framework that optimizes the covariance structure of data features rather than relying strictly on class means. Because the geometric distribution of features is inherently more stable than discrete, noisy labels, the model builds an intrinsic robustness to corrupted data.
In a similar vein of structural adaptation, Hahn et al. introduce SuPerFed. By constraining local updates to a "low-loss subspace" anchored directly to the global model, it allows for personalized learning while preventing the local weight space from drifting into the erratic configurations typically caused by noise memorization.
Key Foundational Work
Dealing with Label Quality Disparity in Federated Learning (2020) — contributed 2 concept elements to this direction.
Robust Federated Learning With Noisy Labels (2020) — contributed 3 concept elements to this direction.