Continual learning systems learn tasks one after another and tend to forget old tasks when trained on new ones. The standard fix is to keep a buffer of old training examples and replay them. That is exactly what privacy rules forbid, and it is also what breaks formal privacy accounting, because each stored example is reused at every later task. The alternative is to replay samples from a generative model instead of real data, but generators trained adversarially are fragile. They collapse onto a few modes, and retraining a generator on its own outputs degrades them further with each task. Adding the gradient noise that differential privacy requires should make that fragility worse.
This note proposes a way out that rests on one accounting observation and one mechanism. The observation is that if each person's data is touched only while their own task is being learned, and only through noised gradients, then everything the system does afterwards with synthetic samples counts as post-processing. The per-person privacy budget then stays fixed however many tasks follow, and no raw buffer is needed. The mechanism is that every tool we use to stop mode collapse should be computed only from the generator's own samples. Such a tool costs no privacy budget, so we can use it as heavily as we like. We use a Wasserstein proximal anchor, a penalty that keeps the generator's output distribution close to its previous state, applied both within a task, to absorb privacy noise, and across tasks, to stop old modes from drifting.
Core Claim
Differential privacy (DP) bounds how much any one person's data can change what a released model looks like. The budget $\epsilon$ measures that bound, and smaller $\epsilon$ means stronger privacy. The existing DP continual-learning methods keep raw data and fight the cost of reusing it. Continual Learning with Differential Privacy samples one task-specific memory block per rehearsal step and cuts budget growth from quadratic to linear in the number of tasks $T$. Lifelong DP achieves a budget that does not grow with $T$. It does this by perturbing the Taylor coefficients of the training objective, but it still stores disjoint raw memory batches, and the method is tractable only for shallow models. The non-private generative replay methods, DDGR and data-free model inversion (Model Inversion with Layer-Specific Modeling and Alignment), drop the buffer but offer no formal guarantee at all.
Our claim has two parts. First, a replay system with no raw buffer can have a budget that is constant in $T$. This requires three conditions: the generator's critic and the classifier see task-$t$ real data only during task $t$; they see it only through clipped, noised per-example gradients; and all later replay uses synthetic samples. The per-person cost is then $\epsilon_{\text{critic}}+\epsilon_{\text{cls}}$ for task $t$, by parallel composition over disjoint task datasets, and it does not grow with $T$. This holds under the assumption that each person contributes to one task. If a person appears in $k$ tasks, the cost composes $k$ times, and we state this explicitly as the known leak for correlated tasks.
Second, the hard part is therefore stability rather than accounting, and it can be addressed at zero privacy cost. In a DP generative adversarial network (GAN), only the critic, the network that scores real against fake, touches private data. The generator only receives gradients through the critic. Any regularizer defined on generator samples is free.
The specific regularizer is the Jordan–Kinderlehrer–Otto (JKO) step studied in A Unifying View of Variational Generative Wasserstein Flows. That paper shows the step stabilizes W1 and MMD-GANs, cutting CIFAR-10 MMD-GAN FID from 49.39 to 15.76 at about 8.6% extra time, but it considers neither privacy nor sequential tasks. Write $p_{G}$ for the generator's output distribution, $\hat D$ for the noised critic's divergence estimate, $W_2$ for the 2-Wasserstein distance, $\tau$ for the step size, and $G_k$ for the generator after the previous outer step. At task $t$, with old-task labels $y<t$ and a frozen copy $G^{(t-1)}$ of the previous task's generator, we solve
$$G_{k+1}=\arg\min_G\ \hat D\big(p_G(\cdot\mid y{=}t),p_{\text{data},t}\big)+\frac{1}{2\tau}W_2^2\big(p_G,p_{G_k}\big)+\frac{\lambda}{2}\sum_{y<t}W_2^2\big(p_G(\cdot\mid y),p_{G^{(t-1)}}(\cdot\mid y)\big).$$
The second term damps each noisy critic-driven step. This is our hypothesis for why it helps under DP: the JKO paper interprets parameter-space JKO as a preconditioned, natural-gradient-like flow, which should average out isotropic noise. The third term replaces "retrain on your own samples" with an explicit distributional anchor for old classes. That targets the copy-of-a-copy degradation that DDGR documents for standard generative replay. Both $W_2$ terms involve only generated samples, so neither consumes $\epsilon$.
Research Program
We first build DP-trained conditional W1 and MMD generators. The per-example clipping and Gaussian noise are applied to the real-data terms of the critic loss, including the gradient-penalty term evaluated at real/fake interpolates, and a moments accountant tracks the budget. We then add the within-task JKO term and the cross-task anchor, computing $W_2$ with minibatch optimal-transport solvers on generator outputs. The classifier is trained with DP-SGD on current-task real data mixed with synthetic replay.
The analysis has three parts. The first is a written privacy proof covering the full pipeline, including the classifier's use of synthetic samples and the generator distillation. The second is an empirical study of how mode coverage degrades as $\epsilon$ shrinks, with and without each stabilizer. The third is a study of how old-task sample quality evolves over long task sequences.
The known JKO open problem, choosing $\tau$, becomes sharper here, because the right $\tau$ should scale with the noise level. We will test a simple rule that sets $\tau$ in proportion to the inverse of the noise multiplier and compare it against a grid search. Every hyperparameter search counts toward the budget: tuning on private data costs $\epsilon$. We therefore either tune on a public proxy dataset or charge the search to the accountant.
Baselines
The naive reference points are fine-tuning with DP-SGD and no replay, which gives the forgetting floor, and naive DP rehearsal. For naive DP rehearsal, Lifelong DP reports 10.43% average accuracy on Permuted MNIST after 20 tasks. The non-private ceilings are A-GEM, which Continual Learning with Differential Privacy reports at about 79.3% on 17-task Permuted MNIST, and GPM, which Gradient Projection Memory reports at 93.91% on Permuted MNIST.
The strongest private competitors are L2DP-ML and DP-CL. L2DP-ML reports 47.73% average accuracy after 20 Permuted MNIST tasks at $\epsilon=0.5$. DP-CL reports about 47.5% after 17 tasks at $\epsilon=0.85$. Both keep raw buffers, so they are strictly advantaged on data access. We run them on the same task orders, the same architectures where their methods allow it, and the same $\epsilon,\delta$. Beating them means higher average accuracy at equal or smaller $\epsilon$ while storing no raw examples. Matching them without a buffer would already be a meaningful result. For Split CIFAR-100, DP-CL reports only that the gap widens with tasks; no reported end-task number was found on Lacuna, so we rerun it.
The strongest structural competitor is one frozen DP generator per task. This removes recursive retraining entirely, and we compare against it at matched total generator parameters. Beating it means equal or better old-task accuracy at a fixed memory budget, or matching it with a single model.
The key ablations remove the within-task JKO term, the cross-task anchor, or both. They also replace JKO with a weight exponential moving average (EMA) of the generator. The EMA comparison isolates whether it is the distributional proximal step that matters.
Why not just…?
Why not just use GPM, which stores no images? GPM projects new-task gradients away from a subspace built from old-task activations. It is excellent at short horizons, with 0.00 backward transfer on 20-split miniImageNet. However, its authors report that the protected subspace fills up over long sequences until the network can no longer learn, and they suggest adding generative replay as the remedy. Its stored bases are raw activation statistics with no DP bound. We run DP-noised GPM, with noise on the activation SVD, on long sequences. If it matches our method at equal $\epsilon$, generative replay is unnecessary at those lengths and we will say so.
Why not just keep one frozen DP generator per task? This is the real threat, because it sidesteps copy-of-a-copy degradation for free. Its cost is memory that grows linearly with $T$, and it does nothing about DP-induced collapse within a task. The matched-memory comparison settles the question. If per-task generators win at equal parameters, the cross-task anchor is not earning its place, and the contribution reduces to the within-task, privacy-free stabilizer.
Why not just average the generator's weights? Weight EMA is also privacy-free and cheap. Close Shortcut Wins Long attributes part of its stability to an implicit EMA effect, though in non-private distillation. If EMA matches JKO under DP noise, we report that the operative principle is "stabilize where the data is not touched," not the Wasserstein geometry specifically. The EMA ablation is designed to show exactly this.
Why not just train a DP diffusion model, which has no minimax game? A diffusion model generates samples by gradually denoising random noise, and it trains with a stable regression loss. Its drawback here is cost: DDGR takes about 75,000 seconds on CIFAR-100 without any privacy machinery. DP-SGD also requires per-example gradients, which multiplies that cost. We include DP-DDGR at a compute budget matched to our method, together with its untruncated cost, to show where each method sits on the accuracy/compute/$\epsilon$ trade-off.
Proposed Experiments
We use 20-task Permuted MNIST for direct comparison with L2DP-ML and DP-CL. We use 20-task Split CIFAR-100 because it is shared with DP-CL, DDGR and model inversion; DDGR's non-private 59.20% there is a ceiling reference, not a target. We use CORe50's 79 batches to stress long-horizon drift. For each $\epsilon\in\{0.5,1,2,8\}$ we report average accuracy, forgetting, and a mode-coverage measure: per-class recall, and the fraction of classes an oracle classifier recognizes in samples. We also report per-class FID of task-1 samples after every subsequent task, and a membership-inference attack on released samples as an empirical check on the accounting.
Three checks are decisive. The first tests whether the within-task stabilizer does its job: coverage should fall off sharply as $\epsilon$ drops without the JKO term, and much less steeply with it. If the two curves coincide, the hypothesis that the proximal step absorbs DP noise is wrong. The second tests whether the cross-task anchor does its job: task-1 FID should stay flat over 20 or more tasks with the anchor and climb without it, mirroring DDGR's reported gap between standard generative replay and diffusion replay (17.31% versus 6.98% forgetting on ImageNet). The third is the headline test: at $\epsilon=0.5$ on Permuted MNIST, the buffer-free method should reach or exceed L2DP-ML's 47.73%.
References
- Continual Learning with Differential Privacy
- Lifelong DP: Consistently Bounded Differential Privacy in Lifelong Machine Learning
- DDGR: Continual Learning with Deep Diffusion-based Generative Replay
- Model Inversion with Layer-Specific Modeling and Alignment for Data-Free Continual Learning
- A Unifying View of Variational Generative Wasserstein Flows
- Gradient Projection Memory for Continual Learning
- Close Shortcut Wins Long: Seeking Diverse and Stable Generators for Data-Free Knowledge Distillation