Differentially Private Federated Class-Incremental Learning for EEG Mental State Classification

Baran Can Gül, Ali Gülümser, Nasser Jazdi, Michael WeyricharXiv 2026 · arXiv

Training machine learning models on sensitive brain data (EEG) presents a difficult trifecta of challenges: you want to keep the data on the user's device (Federated Learning), you need formal mathematical guarantees that the model doesn't leak private info (Differential Privacy), and the model must learn new mental states over time without forgetting old ones (Class-Incremental Learning).

A new paper, "Differentially Private Federated Class-Incremental Learning for EEG Mental State Classification," tackles this "perfect storm" of instability, specifically focusing on the realistic scenario of small groups (like a single household or research cohort) rather than massive web-scale datasets.

TL;DR

The authors introduce FedEEG, a framework that enables wearable EEG devices to learn new mental states across a small federation of users while maintaining strict privacy. By combining "head-scoped" differential privacy with clever weight-consolidation techniques, they achieved a nearly 10,000x improvement in the privacy budget ($\epsilon$) with utility comparable to non-private training.

What Problem Does This Paper Address?

Most research in Federated Learning (FL) and Differential Privacy (DP) assumes "web-scale" cohorts—millions of users where individual noise cancels out. However, wearable EEG applications usually involve small groups ($K \le 10$). In this "small-cohort" regime, standard DP noise is overwhelming, and the lack of user diversity makes models prone to catastrophic forgetting—where learning a new state (like "drowsy") causes the model to completely forget how to recognize an old one (like "concentrating").

Furthermore, standard classification layers (Softmax) fail during incremental learning because they provide no "negative supervision" for classes the model isn't currently seeing, leading to degenerate gradients and model collapse.

Methods

The authors propose FedEEG, which introduces several novel components to stabilize learning:

  1. FedEEG Objective: It combines Synaptic Intelligence (SI)—which tracks which weights are most important for past tasks—with a lagged adaptive FedProx term. The "lagged" part is key: it uses the update norm from the previous round to determine how much to anchor the current update, avoiding circular dependencies.
  2. Masked Binary Cross-Entropy (BCE): Instead of standard Softmax, they use a masked BCE scheme. This ensures that even when the model is only seeing data for one new class, it still receives a signal to maintain the boundaries for previously learned classes.
  3. Head-Scoped DP: This is the paper's most significant privacy "hack." Instead of privatizing the entire 23,000+ parameter model, they only apply DP noise to the classification "head" (the last 325 parameters). Because DP noise scales with the number of dimensions, this drastically reduces the amount of noise needed to achieve a strong privacy guarantee.
  4. Balanced-Batch Replay: To further fight forgetting, devices keep a small, local buffer of past data. Crucially, this data never leaves the device, so it doesn't impact the formal DP guarantee.

Datasets

The study uses a $K=5$ client federation:

  • Bird et al. Dataset: Four subjects providing 4-channel EEG data for three states: neutral, relaxed, and concentrating.
  • Custom MUSE 2 Dataset: An in-house pilot dataset recorded with a consumer-grade headband, adding two more states: alerted and drowsy.

Classes were introduced in stages (e.g., learning neutral/relaxed first, then concentrating/alerted, then drowsy) to simulate real-world incremental use.

Key Insights and Ideas

  • The Dimensionality Curse of DP: In small cohorts, you don't get "privacy amplification" from subsampling. By "scoping" DP to just the classification head, the authors reduced the privacy budget $\epsilon$ by 200x on its own.
  • Softmax is the Enemy of Incremental Learning: The authors proved that under single-class increments, Masked Softmax produces exactly zero gradients. Switching to Masked BCE is not just an optimization; it's a requirement for the model to move at all.
  • Aggregation Weighting Matters: In highly imbalanced groups (where one user has 85% of the data), standard "sample-count" weighting causes the model to overfit to the dominant user. The authors found that $\sqrt{n}$-weighting provided the best middle ground for stability.
Per-class accuracy results showing that most classes significantly exceed chance, though "alerted" struggled due to data starvation.

Key Results

  • Privacy Breakthrough: Combining head-scoping with fewer communication rounds resulted in a total 9,615x reduction in the achievable $\epsilon$. They reached a "reportable" $\epsilon \approx 10.4$, whereas a naive full-model approach would have an $\epsilon$ in the hundreds of thousands (effectively no privacy).
  • Utility: The DP-protected model's performance was not statistically distinguishable from non-private training at their sample size.
  • Forgetting Mitigation: The FedEEG framework reduced the Forgetting Measure (FM) significantly. Adding on-device balanced replay reduced mean forgetting by an additional 56.7%.
  • Classification Accuracy: A "class-scoping" approach (removing the data-starved "alerted" class) yielded 50.5% ± 3.6% accuracy across the remaining 4 classes—roughly 2.02x better than random chance.
Fig. 2. (a) The achievable ε reduction decomposed into two isolated factors, each measured while holding the other fixed: head-scoping alone (at the original 9-round budget) already gives 200×; a further, independent 48× comes from reducing communication rounds. The larger share of the total 9,615× reduction is attributable to head-scoping itself and not to training for fewer rounds. (b) Utility (FM) of the DP-protected 3-round run against its properly matched non-DP counterpart at the identical round budget (paired t-test, p=0.20), and against the original 9-round non-private protocol for context only.

Limitations and Future Work

  • Partial Privacy: "Head-scoped" DP only formally protects the final layer. While the feature extractor is aggregated, it isn't strictly DP-protected, meaning it could theoretically be vulnerable to sophisticated gradient inversion attacks.
  • Coarse Classification: While 50% accuracy is much better than chance, it is sufficient for "coarse" states (like detecting if a driver is drowsy) but not yet for fine-grained clinical diagnostics.
  • Scaling: The results are based on a very small cohort ($K=5$). Future work is needed to see how these dynamics change as the group grows and "subsampling amplification" begins to help the privacy budget.

For more details, you can find the full paper accepted at the TrustFL-SIS workshop at ICDM 2026.

Summary updated

Abstract

Federated learning (FL) under differential privacy (DP) is typically evaluated on large client cohorts, where subsampling and cross-client averaging amplify privacy and smooth injected noise. Personal biosignal applications such as wearable EEG instead involve small federations of a handful of subjects, a regime whose interaction with formal client-level DP is largely uncharacterized. We study differentially private federated class-incremental learning in this small-cohort regime, combining a public EEG dataset with a custom four-channel MUSE 2 recording set into a K=5-client federation, and give a masked per-class training protocol that avoids degenerate gradients and missing negative supervision under class-incremental streaming. We propose FedEEG, combining Synaptic-Intelligence consolidation with a lagged adaptive FedProx term, and show it reduces catastrophic forgetting relative to FedAvg, FedProx, and a federated Learning-without-Forgetting baseline; on-device balanced-batch replay reduces mean forgetting by a further 56.7%. A class-scoping diagnosis yields $50.5\%\pm3.6\%$ accuracy with 6 seeds and 2.02 times chance. We also introduce head-scoped client-level DP, which protects the classification head and alone reduces the achievable $\varepsilon$ by 200 times at a matched round budget; combined with fewer communication rounds it reaches $\varepsilon\!\approx\!10.4$ with utility not statistically distinguishable from non-private training.
BibTeX
@misc{gul2026differentially,
  author = {Baran Can Gül and Ali Gülümser and Nasser Jazdi and Michael Weyrich},
  title = {{Differentially Private Federated Class-Incremental Learning for EEG Mental State Classification}},
  year = {2026},
  eprint = {2611.17944},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.17944},
}
Figure