A continual learner has to pick between two bad options. If it updates shared parameters for each new task, it gains from what earlier tasks taught it but forgets those tasks. If it freezes everything old and adds a fresh module per task, it stops forgetting but also stops sharing. One method sits between these: Additive Parameter Decomposition (Scalable and Order-robust Continual Learning with Additive Parameter Decomposition). It keeps a shared component that keeps learning and repairs each old task's private correction term whenever the shared part drifts. The catch is the cost of that repair, which grows with the number of past tasks, and the authors flag it as a limitation. Meanwhile, systems work shows that the kind of training continual learners run, small-batch training of low-rank adapters, leaves most of the GPU idle. A single LoRA job (training small low-rank matrices added to frozen weights) can occupy as little as 16.7% of an A100's streaming multiprocessors (its parallel compute units) and under 55% of its memory (PLoRA: Efficient Concurrent LoRA Training for Large Language Models).
We propose to pay for retroactive repair with that idle capacity. Concretely, the model learns each new task through one shared low-rank adapter and one task-private adapter. Meanwhile, small "correction jobs" for the old tasks' private adapters run concurrently on the same GPU and the same frozen base weights. Each correction job pulls its old task's behavior back toward a stored snapshot as the shared adapter moves. The claim is that this co-location makes shared-parameter continual learning with per-task compensation affordable at nearly the wall-clock cost of training the new task alone. It should therefore beat frozen-base adapter isolation on forward transfer while matching it on forgetting.
Core Claim
Write the effective weights for task $t$ as
$$W_t = W_0 + A_s B_s + A_t B_t,$$
where $W_0$ is the frozen pretrained weight, $A_s B_s$ is a low-rank adapter shared across all tasks and updated on every new task, and $A_t B_t$ is a low-rank adapter private to task $t$. When task $t$ finishes we store a snapshot $\bar{s}_t$ of the shared adapter, which costs only a few megabytes. While a later task $k$ trains and moves the shared adapter to $s$, each active correction job for an old task $j$ updates only $A_j B_j$ to minimize
$$\mathcal{L}_j = \mathbb{E}_{x}\, D\big(f(x; W_0, s, A_j B_j),\ f(x; W_0, \bar{s}_j, \bar{A}_j \bar{B}_j)\big).$$
Here $f$ is the network output, $\bar{A}_j \bar{B}_j$ is task $j$'s private adapter frozen at the end of its training, $D$ is a divergence (KL for classifiers, action-space mean squared error for robot policies), and $x$ ranges over inputs available at the time. In the default rehearsal-free setting those inputs are the current task's observations.
Two things are new relative to the closest work. First, compensation is concurrent rather than post-hoc. APD applies retroactive updates during new-task optimization by iterating over the old residuals. Our correction jobs track drift step by step as separate kernels, and the hardware cost is absorbed into slack. Second, the systems side changes. PLoRA (PLoRA: Efficient Concurrent LoRA Training for Large Language Models) packs independent adapter jobs with no relationship between them. Here the packed jobs are coupled: each correction job reads the shared adapter that the main job is writing.
Research Program
The work has three parts.
Runtime. We will build a runtime that co-locates one main training job, which updates the shared adapter and the current task's adapter, with up to $K$ correction jobs. All of them reuse one copy of the frozen base. We borrow PLoRA's two ideas: tiling kernels along the sequence and hidden dimensions rather than the small rank dimension, and a knapsack-style packing planner. The planner is changed in one way. It no longer packs a fixed job list. At each scheduling interval it chooses which old tasks get a correction slot, ranked by measured drift: the current value of $\mathcal{L}_j$ estimated on a few probe inputs.
PLoRA identifies linear growth of optimizer-state memory as its main out-of-memory risk. We address this by keeping optimizer states for unscheduled correction jobs in host CPU memory and swapping them in when a job is scheduled. This is the only role we give the host. We do not assume the CPU contributes compute, only capacity. The transfers are asynchronous, in the spirit of the decoupled kernels in PAL – parallel active learning for machine-learned potentials.
Learning dynamics. Correction jobs see a moving target and may lag it. We will measure how forgetting depends on the number of correction slots $K$ and on correction frequency. We will also check whether concurrent tracking is better or worse than APD-style sequential compensation applied over the same total number of correction steps.
Input distribution. Rehearsal-free correction uses current-task inputs. This may constrain old-task behavior poorly when the input distributions differ sharply. We therefore test a variant with a small stored buffer as well. This variant is a real risk in its own right, because Replay can provably increase forgetting proves, and confirms on Rotated and Split MNIST, that replaying a small number of samples can make forgetting worse. We will sweep buffer size explicitly rather than assume more is better.
Baselines
The naive reference is sequential fine-tuning of a single shared adapter with no protection. It gives the forward-transfer ceiling and the forgetting floor. LIBERO reports that this setup out-transfers protective methods while forgetting badly (LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learning).
The strongest existing method on the robot benchmark is CLARE (CLARE: Continual Learning for Vision-Language-Action Models via Autonomous Adapter Routing and Expansion). It freezes a pretrained vision-language-action model, adds adapters when feature novelty crosses a threshold, and routes inputs with small autoencoders. It reports an area-under-the-curve of 75.1±1.3 on LIBERO-Long against 52.9±1.6 for experience replay, with near-zero negative backward transfer. It is the strongest competitor because it already achieves almost no forgetting at about 2% parameter growth per task, so our method can only win on forward transfer. We will run CLARE and our method on the same base model, the same adapter insertion points, and the same CLARE-style router for inference-time task selection. Both get the same per-task training steps and the same hyperparameter search budget, and both are rehearsal-free in the main comparison. Beating it means a higher LIBERO-Long area-under-the-curve beyond the seed spread, with negative backward transfer no worse than CLARE's, and per-task wall-clock time within 15% of CLARE's.
On Split CIFAR-100, the algorithmic ancestor APD reports about 60.7% final accuracy against 53.7% for EWC. We will rerun APD under our setup and compare at equal accuracy on wall-clock time, and at equal wall-clock time on accuracy.
Experience replay with reservoir sampling serves as the replay reference, run at matched buffer size in the buffered variant. GCR (GCR: Gradient Coreset Based Replay Buffer Selection For Continual Learning) reports 2–4% gains over replay on sequential CIFAR. The file gives no per-setting table numbers, so we will rerun it ourselves.
Three ablations isolate the key ingredients. The first runs exactly the same correction objective but time-sliced after each main step rather than co-located. This isolates the systems contribution: identical accuracy is expected, so only wall-clock time should differ. The second removes the shared adapter, giving private adapters only with no correction needed. This isolates the forward-transfer contribution. The third replaces drift-ranked slot selection with round-robin selection.
Why not just…?
Why not just freeze the base and add per-task adapters, as CLARE does? This is the real threat. CLARE's near-zero forgetting at 75.1 area-under-the-curve leaves little headroom on retention. Our bet rests on forward transfer. The LIBERO study finds that protective methods transfer worse than naive fine-tuning, and isolated adapters cannot pass anything learned on task $j$ to task $k$ except through the frozen base. The no-shared-adapter ablation and the forward-transfer metric settle this directly. If forward transfer does not improve over CLARE on LIBERO-Long and LIBERO-Goal, the idea fails, and we will report that.
Why not just mix replay or distillation samples into the main batch? This raises the effective batch size of the new task's update. PLoRA reports that larger batches degrade LoRA quality, which is exactly why small-batch jobs underuse the GPU. It also couples old-task gradients into the shared adapter, the parameter whose drift we are trying to control. Separately, small mixed-in replay can increase forgetting according to the replay-harm result. The matched-buffer replay baseline and the buffer-size sweep test both concerns.
Why not just run APD's retroactive updates sequentially and accept the cost? Accuracy might be the same. Our time-sliced ablation is exactly this control. The claim then reduces to a wall-clock and memory one. It holds only if co-located corrections raise occupancy without slowing the main job, which we measure directly with occupancy counters and main-job step time. If interference between kernels eats the gain, the contribution collapses to the algorithmic variant, and we will say so.
Why not just put correction jobs on a second GPU? That doubles the hardware, and it sends every shared-adapter update across devices. Our comparison holds hardware fixed to one A100, as in PLoRA's evaluation.
Proposed Experiments
The primary setting is LIBERO-Long with 40 sequential tasks, following CLARE's protocol, plus LIBERO-Goal and LIBERO-Object, where LIBERO's forward-transfer differences are largest. The secondary setting is Split CIFAR-100 and Split Tiny-ImageNet with a ViT backbone and LoRA adapters, with the APD and replay comparisons. A third, exploratory setting runs PLoRA's tasks (GSM8K, MRPC, CoLA, WNLI) sequentially on Llama-3 3B. No reported continual-learning result was found on Lacuna for this sequence. Metrics are area-under-the-curve, forward transfer, negative backward transfer, final average accuracy or success rate, per-task wall-clock time, streaming-multiprocessor occupancy, and peak GPU and host memory.
Three checks are decisive. First, at $K=0$ correction slots our method should reproduce sequential fine-tuning's forgetting. As $K$ grows, negative backward transfer should fall toward CLARE's while forward transfer stays near sequential fine-tuning's; if forward transfer falls in step with forgetting, the shared adapter buys nothing. Second, main-job step time with $K$ co-located corrections should stay within 15% of the solo job for $K$ up to the point where occupancy saturates. PLoRA's 16.7% baseline occupancy suggests several jobs fit, but we do not assume a number. Third, the concurrent-versus-sequential comparison at equal total correction steps tells us whether tracking drift online helps retention, hurts it through staleness, or is neutral.
References
- Scalable and Order-robust Continual Learning with Additive Parameter Decomposition
- PLoRA: Efficient Concurrent LoRA Training for Large Language Models
- PAL – parallel active learning for machine-learned potentials
- Replay can provably increase forgetting
- LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learning
- CLARE: Continual Learning for Vision-Language-Action Models via Autonomous Adapter Routing and Expansion
- GCR: Gradient Coreset based Replay Buffer Selection for Continual Learning