Many monitoring systems find unusual data without labels by training an autoencoder on "normal" inputs. An autoencoder is a network that squeezes each input through a narrow bottleneck and tries to rebuild it. Inputs that rebuild badly are flagged as anomalies, and a sustained rise in rebuild error is read as a sign that the data stream has shifted. The trouble begins after a shift is detected. Current systems either throw the autoencoder away and train a new one from scratch, or keep fine-tuning the old one. The first choice discards everything it knew. The second leaves a detector that still rebuilds the previous normal well, so if yesterday's normal is today's fault (a broken sensor mode, a retired operating regime, a class that is now off-limits), the detector waves it through. The detector needs to forget the old normal on purpose, while keeping the general ability to compress inputs that both regimes share.
We propose to treat this as a machine unlearning problem. Machine unlearning means removing what a trained model learned from a chosen subset of data without retraining it from scratch. The setting already supplies the subset we need. Every reconstruction-based drift detector keeps a reference window of pre-drift inputs to compare against, which is a ready-made, label-free forget set. The post-drift buffer it collects before raising an alarm is a ready-made retain set. When the drift test fires, we run a bounded, representation-level unlearning step on the autoencoder itself. The goal is for old-regime inputs to start rebuilding badly again, while rebuild quality on new-regime inputs and on structure shared by both regimes stays intact. We then check whether the forgetting is real or only skin-deep.
Core Claim
A reconstruction-based detector should end a drift episode in a specific state. Old-regime inputs should score as anomalous, new-regime inputs as normal, and the shared low-level features should be preserved. Neither existing adaptation strategy reaches this state. A hard reset loses the shared features. Warm-start fine-tuning keeps the old manifold, which is the region of input space the autoencoder has learned to rebuild well. Our claim is that an unlearning step does reach this state. It is triggered by the detector's own alarm, and it pushes old-window codes in the encoder's middle layers toward an uninformative target while anchoring the retain buffer. The effect should be largest exactly where hard resets are known to fail, namely high-dimensional streams.
Where This Sits in Prior Work
Several papers motivate the gap from the detection side. strAEm++DD detects drift with a Mann-Whitney U test, a rank-based comparison of rebuild errors in a reference window versus a moving window. It adapts by hard reset, and the authors report that on the high-dimensional MNIST-23 stream it fails to return to pre-drift performance. We keep its detection and replace its reset. ABCD adds a variance-aware statistical bound on rebuild error, detects shifts with high F1, and identifies which features changed, but it never adapts the model. We use it as an alternative alarm. S-TRIGGER and CLeaR both use rebuild error as a shift sensor, then work hard to prevent forgetting: S-TRIGGER replays generated past samples, and CLeaR penalizes changes to weights that mattered for earlier regimes. Both treat forgetting purely as a defect, whereas we treat targeted forgetting as the objective. Unsupervised Unlearning of Concept Drift with Autoencoders uses "unlearning" for the opposite operation. It maps drifted inputs back onto the frozen old manifold, which presumes the old normal stays correct forever. Our setting is the one where it does not.
From the unlearning side, UIL is the closest existing combination of unlearning and drift. It replaces sliding-window retraining with an influence-function step that removes the oldest data chunk from a classifier. It is roughly three times faster than retraining and recovers from noise drift in 6.33 chunks rather than 17. Its limits are what our design changes. UIL forgets on a fixed chunk schedule regardless of whether drift happened, it targets a supervised classifier rather than the detector, and it does visibly worse on semantic drift.
The method comes from the WMDP paper's representation-misdirection method. That method steers hidden activations of forget inputs toward a fixed random vector, with a retain loss on benign inputs, and lowered hazardous-knowledge accuracy from 63.7% to 31.2% while MMLU, a broad general-knowledge benchmark, dipped only from 58.1% to 57.1%. An Illusion of Unlearning? constrains the design. It shows that output-level unlearning can leave the "forgotten" class 70–80% linearly decodable from the model's internal features. This tells us an objective that only inflates decoder error is likely to be an illusion, and that we must audit the encoder.
Research Program
The detector is a standard streaming autoencoder with encoder $E$ and decoder $D$. When the alarm fires, we hold two sets: the reference window $R$ of pre-drift inputs and the post-drift buffer $B$. The adaptation objective is
$$ \mathcal{L} = \frac{1}{|B|}\sum_{x \in B} \|D(E(x)) - x\|^2 \;+\; \alpha \frac{1}{|R|}\sum_{x \in R} \|h_\ell(x) - c\, u\|^2 \;+\; \beta \frac{1}{|B|}\sum_{x\in B} \|h_\ell(x) - h^{0}_\ell(x)\|^2 . $$
Here $h_\ell$ is the encoder's activation at a middle layer $\ell$, $h^0_\ell$ is the same activation in the frozen pre-update model, $u$ is a fixed random unit vector, $c$ is a scale, and $\alpha$ and $\beta$ weight the forget and anchor terms. The first term learns the new normal. The second term destroys the old regime's internal code rather than merely its output. The third term keeps shared structure from drifting. All three are bounded, unlike gradient ascent on rebuild error, which is the usual route to collapse.
Unlearning runs only when the old regime is declared obsolete. In the role-reversal setting this is a property of the task. In general it is an operator decision, and we study the case where it is wrong under recurrent drift.
The program has three parts. We build the alarm-triggered unlearning step and compare it against reset and warm-start fine-tuning. We audit whether forgetting is genuine by probing encoder features and measuring how fast the old regime comes back. Finally, we characterize when unlearning hurts, namely when the old regime recurs as legitimately normal.
Baselines
The naive reference point is the hard reset of strAEm++DD. After the Mann-Whitney alarm, a new autoencoder is trained from scratch on $B$. This baseline is a genuine threat on low-dimensional streams, where its G-mean recovers to 0.8–0.9 almost immediately on Sea and Circle. G-mean is the geometric mean of the recall on normal and on anomalous points. We report that number from the paper and also reproduce it. On MNIST-23 there is no reported result found on Lacuna for a numeric G-mean, only the qualitative finding that it does not return to pre-drift levels. Our method beats this baseline if it recovers faster on high-dimensional streams while matching it on Sea and Circle.
The second baseline is warm-start fine-tuning on $B$ with the same number of gradient steps. It isolates whether the forget term matters at all.
The third baseline is a scheduled version of UIL applied to the autoencoder. It removes the oldest chunk on a fixed schedule, using either its influence-function step or our objective without the alarm trigger. This separates the value of drift-triggered timing from the value of unlearning itself.
The strongest competitor is a dual-model contrast detector. It keeps the old autoencoder frozen, trains a new one on $B$, and scores each input by new-model error minus old-model error, so that old-normal inputs are flagged explicitly.
Two ablations isolate the key ingredient. The first applies the forget term at the output, as a hinge pushing old-window rebuild error above the new threshold, instead of at layer $\ell$. The second removes the anchor term.
All methods share the same detector, windows, buffer size, gradient-step budget, and a 20-configuration hyperparameter search tuned on one held-out stream per family.
Why not just…?
Why not just reset the autoencoder after each alarm? On low-dimensional streams a reset may be as good as anything, and we expect to tie there. The evidence against it is high-dimensional, from the MNIST-23 failure in strAEm++DD. A reset discards the low-level features both regimes share, and a small post-drift buffer cannot rebuild them. If our method does not beat reset on MNIST-23 and Fashion-MNIST role-reversal streams, the core claim fails.
Why not just fine-tune on the post-drift buffer and let old knowledge fade? This works only if autoencoders forget old regimes on their own. They often do not, because they tend to generalize to nearby inputs and rebuild them well. The bet is testable directly: we measure how well warm-start fine-tuning detects old-regime inputs after it has run. If old-regime error rises enough without any forget term, the forget term is unnecessary and we will say so.
Why not just keep the old model and score by contrast? This is the most serious alternative. For a single drift it may match us, since it flags old-normal inputs by construction. Its costs appear over many drifts: it needs one stored model per regime, and its scores become ambiguous when regimes overlap. We test it on streams with five or more successive drifts under a fixed memory budget. If it still matches us there, unlearning buys only memory savings, and we will report the result in those terms.
Why not just map new inputs back to the old manifold? This is what Unsupervised Unlearning of Concept Drift with Autoencoders does. It assumes the old normal remains the target. In role-reversal streams that assumption is exactly wrong, and even on its own occlusion task it recovered accuracy only to 77% from 98%. We include it on the role-reversal streams for completeness.
Proposed Experiments
We use the strAEm++DD streams Sea, Circle, MNIST-01 and MNIST-23, keeping their alarm settings. We add role-reversal streams built from MNIST and Fashion-MNIST: one class is normal before drift, and after drift a different class is normal while the old class appears as the anomaly at rates down to 0.1%. We also use the GAS sensor-drift stream, with ABCD as an alternative alarm. ABCD's reported detection F1 is 0.90, and we use that figure to confirm the alarm works before measuring adaptation.
The metrics are post-drift G-mean, steps to recovery, and the area under the ROC curve for detecting old-regime inputs as anomalies. We also track the false-alarm rate on new-normal inputs, wall-clock time per adaptation, and peak memory over multi-drift streams.
The decisive checks are three. First, on high-dimensional role-reversal streams, our method must beat both reset and warm-start fine-tuning on old-regime detection without raising false alarms on the new normal. Second, an illusion audit must show the forgetting is real. We train a linear probe to separate old-window from new-buffer inputs using frozen encoder features. We also measure resurgence: how many decoder-only gradient steps on old data restore its rebuild error, compared with a freshly trained autoencoder. The middle-layer forget term should show lower probe accuracy and slower resurgence than the output-hinge ablation, following the pattern An Illusion of Unlearning? found for classifiers. Third, on recurrent streams where the old regime legitimately returns, we measure the cost of having forgotten. Unlearning should make the detector re-alarm quickly on the return. If the recovery after that re-alarm is slower than warm-start fine-tuning, the method should be restricted to settings where obsolescence is declared rather than inferred.
References
- Autoencoder-based Anomaly Detection in Streaming Data with Incremental Learning and Concept Drift Adaptation
- Adaptive Bernstein change detector for high-dimensional data streams
- S-TRIGGER: Continual State Representation Learning via Self-Triggered Generative Replay
- CLeaR: An adaptive continual learning framework for regression tasks
- Unsupervised Unlearning of Concept Drift with Autoencoders
- Unlearning-based sliding window for continual learning under concept drift
- The WMDP Benchmark: Measuring and Reducing Malicious Use with Unlearning
- An Illusion of Unlearning? Assessing Machine Unlearning Through Internal Representations
- Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms