Inhibit, Then Consolidate: Gated Unlearning for Language Models

Erasing a fact from a large language model usually damages everything else the model knows. Methods that push the model's probability of the unwanted text downward also tend to break its grammar, its reasoning and its general knowledge, sometimes until it repeats one token over and over. This failure is called catastrophic collapse. It happens because the update is applied to weights that the forgotten fact shares with nearly every other skill. The brain handles a similar problem differently. Subcortical structures such as the basal ganglia do not delete what the cortex has learned. They select behaviour by suppressing particular pathways and releasing the rest, and that selection is later written into cortical habits. We propose to copy this sequence. First, train a small, sparse, input-dependent "inhibitory gate" that blocks the forgotten content only when that content is actually being used. Then use the gate's behaviour to decide which base-model weights may change, and distill the suppression permanently into those weights.

The central bet is about localization. Current localization methods compute one fixed mask over the weights. Our method instead asks which internal channels the forget content passes through, and only on forget inputs. If that input-conditional picture identifies the forget content more cleanly, the permanent edit can be smaller and the model's retained abilities should survive better at equal forgetting. The proposal succeeds or fails on that one measurable claim.

Background and Gap

The unlearning side supplies the problem and the strongest tools. Negative Preference Optimization replaces unbounded gradient ascent with a sigmoid-bounded objective whose gradient fades as the forget text becomes unlikely. This removes the divergence that drives collapse. It does not, however, keep the update off shared weights. WAGLE attacks that second problem. It sets unlearning up as a two-level optimization: forget the targeted data at the upper level while keeping performance on retained data at the lower level. From this it derives a closed-form attribution score for each weight and updates only the most influential ones. Its finding that forget-relevant weights sit in early-to-middle-layer MLP blocks (the feed-forward sublayers) and in attention value/output projections motivates our search space. WAGLE's mask is static, though. A weight is either editable or frozen for every input, and the unlearning gradient still reaches that weight on retain inputs that happen to use it. Our gate is meant to separate the two cases.

The neuroscience side supplies the design principle. We use it as a structural prior, not as a claim about biology. The "Gearbox" hypothesis and related basal-ganglia modelling (the mouse cortico-basal ganglia-thalamic network) describe selection through selective inhibition of specific thalamocortical targets, leaving cortical content intact. Transthalamic routing studies (subcortical circuits mediate communication between primary sensory areas) show that information flow between cortical areas is gated by a separate structure rather than by changing the areas themselves. Dayan's Palmistry argues that purely cortical predictive systems lack a mechanism to decide what not to represent, and that a gating system outside the representation fills this role. That is the same capacity unlearning needs. Finally, the reinforcement-trained visual-motor spiking network shows subcortical reward signals reshaping cortical representations over training. This motivates the second, "consolidation" stage: the gate does more than filter outputs at run time, because it also teaches the base weights.

The research file's survey of prior work at this exact intersection (its section 2) was truncated. We therefore could not verify whether brain-inspired gating has already been applied to unlearning. The closest work we can confirm is WAGLE's static attribution mask, and the contrast drawn here is with that.

Core Claim

A sparse inhibitory gate over MLP channel groups can be trained to fire on forget inputs and to stay almost silent on retain inputs. Let $s_\phi(x) \in [0,1]^K$ be the gate's suppression strength over $K$ channel groups for input $x$, applied multiplicatively as $(1 - s_\phi(x))$ to the frozen base model's MLP activations. The gate is trained with

$$\mathcal{L}_{\text{gate}} = \mathcal{L}_{\text{NPO}}(D_f) + \beta\,\mathrm{KL}_{D_r} + \lambda_r\,\mathbb{E}_{D_r}\|s_\phi\|_1 + \lambda_f\,\mathbb{E}_{D_f}\|s_\phi\|_1 .$$

Here $D_f$ and $D_r$ are the forget and retain sets, and $\mathrm{KL}_{D_r}$ is the divergence from the original model on retain data. The two sparsity weights do different jobs. The term weighted by $\lambda_r$ makes the default state pass-through, which mirrors tonic inhibition being released everywhere except at selected targets. The term weighted by $\lambda_f$ forces the selection to be narrow.

The consolidation stage then takes the channel groups $C$ that carry most of the gate's mass on $D_f$. It updates only the base weights that feed and read those groups, distilling the gated model into the base model on both $D_f$ and $D_r$. After this the gate is discarded.

We predict that, at matched forget quality, the consolidated model keeps more general utility than a WAGLE-masked model, because $C$ comes from routing that depends on the input rather than from a single global attribution score. We also predict that the gate's firing rate on held-out retain data will predict utility loss in advance, giving a cheap early warning of collapse.

Research Program

We first build the gate. It is a small network that reads the residual stream (the running hidden state passed between layers) at a few early-to-middle layers and outputs suppression for MLP channel groups in the same layers. We restrict it to the layers WAGLE identifies, and we sweep group granularity from single neurons to blocks of 64 channels.

Second, we characterize the gate on its own. We measure how often it fires on forget versus retain data, how its suppression is distributed over layers, and how much its selected channels overlap with WAGLE's top-attributed weights. Low overlap combined with better utility is the result that would show input-conditional localization finds something the static score misses.

Third, we run consolidation. We sweep the size of $C$ and compare distilling into the gate-selected weights, into WAGLE-selected weights, and into all weights.

Fourth, we probe residual knowledge. We run a relearning test: briefly fine-tune each final model on a small slice of $D_f$ and measure how fast the forgotten answers return. This matters because a gate alone, or a shallow edit, may hide knowledge without removing it.

Baselines

The naive reference points are the original, unedited model and plain gradient ascent. Gradient ascent is the standard demonstration of collapse and fixes the bottom of the utility scale. We also include Gradient Difference, which adds a retain loss. WAGLE reports that on WMDP it collapses general utility to 0.4459 (WAGLE).

Dense NPO (Negative Preference Optimization) tests whether the bounded objective alone already does the job. WAGLE reports dense NPO at an Unlearning Efficacy of 0.9581 on TOFU with Llama2-7b. The file is truncated before the NPO paper's own numbers, so no reported result was found on Lacuna for NPO's original TOFU figures.

The strongest competitor is WAGLE combined with NPO. It is the most direct existing answer to our question, namely restricting the unlearning update to the weights that matter. The file reports it at an Unlearning Efficacy of 0.9599 on TOFU, and at an average Unlearning Efficacy of 0.6871 with general utility of 0.5530 on WMDP, both on Llama2-7b (WAGLE).

All methods use the same base model, forget and retain splits, NPO objective and retain-divergence weight. Each method gets the same number of optimizer steps and the same hyperparameter-search budget. Our method's budget includes the gate-training steps, so it gets no extra compute. Hyperparameters are chosen on a held-out retain split, never on the test metrics. Beating WAGLE+NPO means two things. On WMDP, general utility must be clearly above 0.5530 at a forget efficacy at least equal to 0.6871. On TOFU, retain accuracy must be higher at matched efficacy near 0.96. In both cases the margin must exceed the variation across seeds.

Three ablations isolate the key ingredient. The first, "distill without gate," distills from an NPO-trained teacher into all weights; it tests whether consolidation alone explains any gain. The second, "gate without consolidation," keeps the gate at run time; it tests whether consolidation adds anything beyond output-level hiding. The third keeps the full pipeline but replaces the gate's channel set $C$ with a random set of equal size, and separately with WAGLE's set.

Why not just…?

Why not just use WAGLE's static mask with NPO? It is the real threat, and it may be enough. If forget-relevant weights are already well separated by a single global score, the gate adds cost without benefit. The experiment that settles this is the swap ablation above: identical consolidation, with only the source of $C$ changing between gate-derived and WAGLE-derived. If WAGLE's set matches ours on utility at matched forgetting, the input-conditional claim is false and we will report that.

Why not just train a LoRA adapter (a small low-rank add-on to the weights) with NPO and merge it? A low-rank adapter restricts the rank of the update, not which inputs it affects. It still changes the computation on every retain input that passes through the edited weights. Our gate's explicit sparsity term on retain data targets exactly that leakage. The "distill without gate" ablation, run with a low-rank adapter in place of full weights, measures whether rank restriction already gives the same retain utility.

Why not just add a refusal filter or keep the gate at deployment? Run-time suppression leaves the knowledge in the weights. Anyone who removes the filter, or fine-tunes briefly, gets it back. This is why the gate is a teacher and not the final product. The relearning test compares gate-only, consolidated, and WAGLE models by how quickly forgotten answers return. If consolidated models recover as fast as gate-only ones, consolidation is cosmetic, and the proposal weakens to a better localization tool rather than a better unlearning method.

Proposed Experiments

All experiments use Llama2-7b on TOFU (fictitious author profiles at the standard forget fractions) and WMDP (hazardous-knowledge multiple choice), because these are the settings with reported numbers in the file. TOFU is scored with Unlearning Efficacy and retain-set accuracy. WMDP is scored with Unlearning Efficacy and the general utility score reported by WAGLE. We add collapse diagnostics: perplexity on general text, rate of repeated n-grams in free generation, and the relearning-recovery curve.

The decisive checks are three. First, whether gate-derived channel sets beat WAGLE-derived sets under identical consolidation, compared at matched forget efficacy. Second, whether the gate's firing rate on held-out retain data correlates with the consolidated model's utility drop across hyperparameter settings. A strong correlation would make the gate a practical collapse predictor even if headline gains are small. Third, whether consolidated models resist relearning at least as well as WAGLE+NPO. A negative result on the first check and a positive one on the second would still be worth reporting: a cheap, input-level diagnostic of when unlearning is about to damage shared skills.