Unlearning in the Null Space of Competitive Networks

When a trained model is asked to forget something, such as one class of images, one person's records, or one hazardous topic, the edit usually damages much more than the target. The standard explanation is entanglement: the same neurons and directions that carry the forgotten knowledge also carry everything else, so any update that erases one disturbs the other. This proposal asks whether that damage is partly an artifact of dense activations. In an ordinary network, a large fraction of neurons fire on every input. In a network with lateral inhibition, by contrast, units in a layer compete and only the few strongest are allowed to fire. The inhibition is implemented as k-winners-take-all (k-WTA), which keeps the top $k$ activations in a layer and zeroes the rest.

Our claim is that competition makes clean forgetting geometrically possible. If activations on the data we want to keep occupy only a small subspace of each layer, then there is a large set of weight directions that the kept data cannot "see." An unlearning update confined to those directions can erase the target at the level of internal features, not just at the output, with almost no first-order damage to everything else. In dense networks the kept data fills nearly the whole activation space, so no such room exists. The concrete prediction is that, at a matched budget, projected unlearning in a k-WTA network should approach the result of retraining from scratch. That means the forgotten class's internal features stop being linearly decodable while accuracy on retained data is preserved. The same projection applied to a dense network should not achieve this.

Core Claim and What Is New

The mechanism comes from continual learning, where it was used for the opposite purpose. Sparsity and Heterogeneous Dropout for Continual Learning in the Null Space of Neural Activations showed that k-WTA lowers the rank of each layer's activation subspace. That leaves a larger null space for Gradient Projection Memory, a method that confines new-task weight updates to directions orthogonal to old-task activations. On 20-task Permuted-MNIST this improved backward transfer from −14.70 to −1.62. That paper only adds tasks and never removes them. The older result in Compete to Compute points the same way: in its Split-MNIST test, a sigmoid network's error on the first task rose to 57.84% after training on the second, while a local winner-take-all network held 6.12%. We propose to invert the mechanism. The unlearning update is projected into the null space of retain activations, and we test whether competition is what makes that null space large enough to hold a real erasure.

Concretely, consider a layer with weight matrix $W$ and input activations $A_r$ collected on a buffer of retain data. Let $U_r$ be the top singular vectors of $A_r$ that capture a fraction $1-\epsilon$ of its energy. Any update to the forget objective is then projected as

$$ \Delta W \leftarrow \Delta W \,(I - U_r U_r^{\top}), $$

so that, to first order, the layer's outputs on retain inputs do not change.

In a k-WTA network, $A_r$ is nonzero only on units that ever win on retain data. Its rank is therefore bounded by the number of such units. The null space is close to coordinate-aligned: it consists mostly of the incoming weights from units that win on forget inputs but rarely on retain inputs. This gives a testable signature. The projection should behave like a soft, data-driven restriction to "forget-exclusive" units. It should be softer than ablating those units, and it should be measurably different from what the same projection does in a dense network.

The forget objective itself targets internal features, not just the output head. This matters because An Illusion of Unlearning? showed that many methods reach 0% output accuracy on the forgotten class while linear probes on frozen features still recover about 70–80% on CIFAR-10. We therefore combine negative-gradient loss with that paper's class-mean feature alignment, which pulls forget features toward retain class means, and we apply the projection to every edited layer.

Relation to Prior Work

Three lines of work already connect sparsity to unlearning, and each differs from this proposal in a specific way.

Unlearning via Sparse Representations inserts a discrete key-value codebook after a frozen encoder and forgets by masking the keys that forget inputs retrieve. It reaches 0% forget accuracy and even a +0.36% retain gain on CIFAR-10 with a ViT, at a reported 20× compute saving over SCRUB. Its sparsity, however, is a fixed nearest-neighbour lookup at a single bottleneck. Its authors also report that it fails when the target is fine-grained, because similar items share keys. We put competition in every hidden layer of a continuous network, and we edit weights by projection rather than deleting slots. This is what we expect to matter when the forget target sits next to a closely related retained concept.

The second line uses sparse autoencoders, which are external networks trained to decompose a model's activations into sparse, interpretable features. SAEs Can Improve Unlearning, CRISP, and SSPU apply these autoencoders to a dense model and use them to clamp, suppress, or project features. These papers motivate our proposal, since they show that sparse coordinates make unlearning more surgical. They also constrain it, because they work and so set the bar. Our difference is that the sparsity lives inside the model's own forward pass, so no external autoencoder has to exist or cover the concept. Hebbian Learning through the Lens of Sparse Autoencoders gives a reason to expect internal competition to produce similar features: it shows that competitive anti-Hebbian suppression behaves like a sparsity penalty.

The third line consists of Winner-Take-All Autoencoders and Compete to Compute. These supply the activation operators we reuse: top-k with lifetime sparsity, and local winner-take-all blocks. They also supply a warning. If $k$ is too small, units become monolithic detectors, which could make a concept and its neighbours share winners. We treat $k$ as an experimental variable for this reason, not as a tuning detail.

Research Program

The first stage builds matched model pairs. We train ResNet-18 and a small ViT on CIFAR-10, CIFAR-100 and Tiny-ImageNet in two forms: standard ReLU/GELU, and k-WTA in every block. We sweep $k$ from roughly 2% to 25% of channels and also include a local winner-take-all variant. We require clean accuracy within about one point of the dense model before comparing unlearning. If competitive networks cannot reach parity at any $k$, that cost is reported as a primary finding.

The second stage measures geometry before any unlearning. For each layer we record the effective rank of retain activations at several values of $\epsilon$. We also record the overlap between forget-class winners and retain winners. This tests whether the null-space advantage seen in continual learning transfers to a class-removal setting.

The third stage runs unlearning. The forget objective is the same throughout: negative gradient plus class-mean feature alignment. We run it with and without projection, on both architectures, and compare against unit ablation and the external-autoencoder and codebook methods described below.

The fourth stage is an exploratory language-model pilot on TOFU, a benchmark of fictitious author profiles that the model is fine-tuned on and then asked to forget. Top-k activation is retrofitted into a few middle MLP layers of the model during the TOFU fine-tuning stage, followed by projected unlearning on the 1%, 5% and 10% splits. TOFU's own baselines either failed to forget at 1% or became unusable at 5–10%, so even a partial gain would be informative. We will drop this pilot if the retrofit costs more than a few points of retain-set ROUGE before any unlearning.

Baselines

The naive reference points come first. One is the dense network after negative-gradient unlearning without projection. The other is the retrain-from-scratch oracle trained on retain data only, which defines the target for both retain accuracy and probe accuracy on the forget class. Head-only retraining is included because An Illusion of Unlearning? shows it gives perfect output-level forgetting with fully intact features. It calibrates how misleading output metrics alone can be.

The strongest existing method on our exact question, removing feature-level knowledge of a class from a vision classifier, is negative gradient plus class-mean feature alignment on a dense network. The file reports 41.54% feature-level forget accuracy for this method on CIFAR-100 with 10 classes forgotten, against 57.43% for plain negative gradient (An Illusion of Unlearning?). We rerun it under that paper's protocol: same backbone, same forget classes, same retain buffer, same number of unlearning epochs, and the same 30-trial hyperparameter search. Beating it means a clearly lower probe accuracy on the forget class, closer to the oracle, while retain accuracy stays within one point of the oracle. The codebook method from Unlearning via Sparse Representations is run with the same frozen encoder for the ViT setting. At the output level it already reaches 0% forget accuracy, so the comparison there is retain accuracy, probe accuracy, and fine-grained forgetting. For the SAE route we train a sparse autoencoder on the dense model's penultimate and middle layers and apply SSPU-style subspace projection (SSPU), with autoencoder training compute counted against its budget. The file reports SSPU only on language models, so no reported result was found on Lacuna for SSPU on CIFAR.

Three ablations isolate the key ingredient. Dense network plus the identical projection tests whether projection alone does the work. A dense network trained with an L1 activation penalty, matched to the k-WTA network's activation sparsity, tests whether competition matters or any sparsity would do. A k-WTA network without projection tests whether the architecture alone is enough.

Why not just…?

Why not just apply the null-space projection to an ordinary dense network? This is the real threat. If dense-plus-projection matches k-WTA-plus-projection, then competition adds nothing and the claim fails. The continual-learning evidence suggests dense networks exhaust their null space quickly (Sparsity and Heterogeneous Dropout), but saturation under many sequential tasks is not the same as saturation under a single retain set. We settle this directly. We report effective retain rank at matched $\epsilon$ for both architectures, and we plot forget-probe accuracy against retain accuracy as $\epsilon$ varies. The proposal survives only if the k-WTA curve dominates.

Why not just ablate the units that fire on the forget class? In a k-WTA network this is cheap and is close to what the codebook method does. We expect it to work for coarse classes. We expect it to fail when the forget and retain concepts share winners, as with a fine class inside a CIFAR-100 superclass, because deleting a shared unit takes retained knowledge with it. That is the topical collateral damage CRISP documents in language models. Projection keeps a unit's retain-relevant input directions intact. If ablation matches projection on the fine-grained split, we will report that the simpler method suffices.

Why not just use an external sparse autoencoder on the dense model? The autoencoder methods work well on language models; for example, SAEs Can Improve Unlearning reports 29.64% on WMDP-Bio against 50.00% for RMU on Gemma-2-2B. They are limited by autoencoder coverage, and inference-time clamping leaves the weights untouched. Our SSPU-style baseline with training cost charged tests whether internal competition matches external decomposition at lower total cost. A tie at lower cost would still be a useful result. A loss would mean internal sparsity is not a substitute.

Proposed Experiments and Decisive Checks

The primary task is single-class and ten-class forgetting on CIFAR-10, CIFAR-100 and Tiny-ImageNet. Metrics are output forget accuracy, retain and test accuracy, and linear-probe accuracy on frozen penultimate and middle-layer features, all compared against the retrain oracle. Robustness is measured by the number of fine-tuning steps on a small labelled forget-class sample needed to recover 50% accuracy.

The harder task is fine-grained forgetting. One CIFAR-100 fine class is removed, and accuracy on its superclass siblings is the collateral-damage metric. This is where we expect competition to separate from both ablation and codebooks, and where the warning about small $k$ in Winner-Take-All Autoencoders predicts a U-shaped dependence on $k$.

The decisive check is one comparison: k-WTA-plus-projection against dense-plus-projection at matched clean accuracy, matched $\epsilon$ and matched tuning budget. Success means forget-class probe accuracy falls to within a few points of the oracle while sibling and retain accuracy stay within one point. Falsification has three possible forms: dense-plus-projection matches it, L1-sparse dense networks match it, or the k-WTA advantage disappears once the measured retain rank is equalized. The last outcome would mean the benefit is entirely explained by rank, and the simplest framing would then be "reduce retain rank by any means."

The TOFU pilot is reported separately, against TOFU's own baselines on forget quality and model utility. No reported result was found on Lacuna for a competitive-activation model on TOFU.