Energy-Conserving Unlearning: Discrete-Gradient Steps That Keep Retain Loss Fixed Across Sequential Forget Requests

When a language model is asked to forget something, such as one person's records or a retracted claim, the standard tools push the model away from the unwanted text. As a side effect, the model also slowly loses general ability. One forget request barely shows this. After a handful of requests the model breaks down. This proposal treats that slow loss as the same phenomenon numerical analysts call energy drift. A physics simulator that only controls its per-step error lets the system's total energy creep up or down over thousands of steps, because each step's small error has the same sign. Geometric integrators stop the creep by building the conserved quantity into the update rule itself. We propose to do the same for unlearning. Each update step is computed so that the model's loss on a fixed set of "keep" data is conserved exactly, to solver tolerance, rather than approximately. We use a discrete gradient, which is an averaged gradient along the step whose defining property makes the conserved quantity unchanged.

The claim is specific and falsifiable. Near a trained model, where the keep-data loss sits close to a minimum, the obvious fix is to project each forget step orthogonal to the keep-loss gradient. That fix still raises the keep loss by a positive second-order amount on every step, so the drift accumulates in one direction across steps and across requests. A discrete-gradient step removes this accumulated term. We test this in an evaluation designed around the weakness of single case studies. Most unlearning results come from one run on one redundant forget set. We instead run many independent single-record forget trials and report paired distributions, together with paraphrase tests that separate genuine forgetting from suppression of one phrasing.

Background and the Gap

Negative Preference Optimization (NPO) supplies the forgetting objective we build on. Negative Preference Optimization replaces gradient ascent's unbounded loss with a sigmoid of the log-likelihood ratio against a frozen reference model. Its gradient therefore fades as the forgotten text becomes unlikely, and the paper shows collapse is exponentially slower than under gradient ascent. Simplicity Prevails removes the reference model and adds length normalization, producing SimNPO. On TOFU, SimNPO reaches Forget Quality 0.99 against NPO's 0.79. The same paper concedes that these methods still work by pushing parameters apart, with no guarantee against loss of utility. We keep NPO and SimNPO as the forgetting direction and change only how each step is taken.

Three results motivate the proposal.

  • Utility collapses under sequential requests. MUSE shows utility collapsing after a few consecutive unlearning requests. It reports that unlearning Harry Potter could erase up to 100% of utility on unrelated tasks without heavy regularization.
  • Forgetting is often surface suppression. Can LLMs Truly Forget Scientific Claims? shows forgetting is often surface suppression. Under Gradient Difference, accuracy on the targeted questions fell from 97.54% to 71.70%, while accuracy on paraphrases of the same claims barely moved (93.92% to 93.51%). The same paper finds NPO without a retain term dropped MMLU from about 58 to about 39.
  • Benchmark success may rest on redundancy. The coreset effect paper finds that 5% of the WMDP and MUSE forget sets does the work of 100%. Success on these benchmarks may therefore depend on redundancy that real single-record requests lack.

Geometric integration supplies the method. The averaged vector field construction guarantees that a scalar is conserved to machine precision for conservative dynamics. UEPI proves that energy-preserving integrators decompose into a discrete gradient plus a skew-symmetric correction. On Hénon-Heiles it held trajectory error at 9.53×10⁻³, where a soft-penalty physics-informed network diverged to 1.50. That contrast between hard structure and soft penalties is exactly the one we expect to see between our method and a retain-loss term. Hamiltonian Neural Networks and SympNets build conservation into learned physical dynamics, and RO-HNN scales it to 600 degrees of freedom.

All of this prior work conserves the energy of a simulated physical system. None of it treats a network's own parameter updates as the trajectory whose invariant must be protected. Conversely, no unlearning paper diagnoses its utility loss as accumulated, one-signed drift.

The clinical case-report literature supplies the evaluation lens rather than a method. Reports such as the 5-FU esophageal case and vertebral artery dolichoectasia are explicit that one striking outcome is an existence proof, not an estimate of the effect. We adopt that discipline. A method is judged by its distribution over many independent single-record trials, not by one headline run.

Core Claim

Let $\theta$ be the model parameters. Let $R(\theta)$ be the keep-data loss on a fixed anchor batch drawn from the retain distribution. Let $g_f$ be the NPO or SimNPO gradient on the forget request.

First-order projection steps along $-h\,(I - \hat{r}\hat{r}^\top) g_f$, where $\hat{r}$ is the unit gradient of $R$ at $\theta$ and $h$ is the step size. Its change in keep loss is

$$R(\theta_{n+1}) - R(\theta_n) = \tfrac{h^2}{2}\, v^\top \nabla^2 R\, v + O(h^3),$$

where $v$ is the projected direction and $\nabla^2 R$ is the Hessian of $R$. Near a trained model this Hessian is close to positive semidefinite, so the second-order term is nonnegative at every step. Over $n$ steps it sums to a secular drift of order $n h^2$, the unlearning analogue of a Runge-Kutta simulator gaining energy.

Our step instead uses the discrete gradient

$$\bar{\nabla} R(\theta_n,\theta_{n+1}) = \int_0^1 \nabla R\big((1-\tau)\theta_n + \tau\theta_{n+1}\big)\,d\tau.$$

By the fundamental theorem of calculus, this satisfies $R(\theta_{n+1}) - R(\theta_n) = \bar{\nabla}R^\top(\theta_{n+1}-\theta_n)$. Projecting $g_f$ orthogonal to $\bar{\nabla}R$, rather than to $\nabla R(\theta_n)$, makes the right-hand side zero. The anchor loss is then conserved up to quadrature and fixed-point tolerance, with no accumulating term.

The prediction follows. The two steps are indistinguishable for one short request. They separate as the number of steps and sequential requests grows, and they separate more at larger step sizes.

Research Program

Building the step. The step is implicit, because the averaged gradient depends on the endpoint. We solve it by fixed-point iteration:

  1. Start from the first-order projected step.
  2. Evaluate $\nabla R$ at two or three Gauss-Legendre points along the segment.
  3. Re-project $g_f$ and repeat until $|R(\theta_{n+1}) - R(\theta_n)|$ falls below a tolerance.

We expect two to three iterations, costing roughly three to six extra anchor-batch backward passes per step. All comparisons are therefore matched on wall-clock and gradient evaluations, not just step count.

Protecting several capabilities. To guard against the model preserving one number while losing ability elsewhere, we conserve a small vector of anchor losses, one per retain domain. These are, for example, TOFU real-authors and world-facts sets, and a general text slice. The step projects onto the orthogonal complement of their discrete gradients.

Scoping what is conserved. Unlike UEPI, we conserve a chosen scalar rather than learn an integrator, so no learned skew-symmetric correction is needed. The skew-symmetric view still tells us the family of admissible steps. The choice of anchor set is the main design risk, and we study it directly.

Analysis. We will measure the realized second-order drift term per step, to confirm that the one-signed drift mechanism is actually present in 7B models. If the Hessian along forget directions turns out to be indefinite, so that projection drift averages out, the core premise fails. We will report that as a finding.

Baselines

Reference points. The naive reference points are the untouched original model and, on TOFU, the retrained-from-scratch model that defines Forget Quality. Every method is placed between these two.

Retain-penalized objectives. The standard practical competitor is NPO and SimNPO with a retain-set cross-entropy term. On TOFU, SimNPO reports Forget Quality 0.99 (Simplicity Prevails). On MUSE-News it reports KnowMem 39.65 against NPO's 37.58. It is the strongest competitor because it already reaches near-retrain forget quality in the single-request setting. Any gain we show must come from the sequential and single-record regimes, not from beating it where it is already saturated. We run it from the released benchmark models on identical forget and retain data, with its retain weight and learning rate tuned on the same validation split and the same number of trials as our method. We will also give it our anchor batches as its retain data, so it sees identical keep information.

First-order projection (the key ablation). Projection orthogonal to the pointwise keep-loss gradient is the ablation that isolates the discrete gradient. It uses the same anchors, forget objective and schedule, and differs only in which gradient is projected out.

Smaller step size. NPO+RT and first-order projection at a reduced step size, with a proportionally larger step count, form a compute-matched control.

What counts as beating them. For sequential requests, no per-request figures are reported in the file (no reported result found on Lacuna for per-request sequential utility curves), so we generate all curves ourselves. Beating the baselines means the following. After a fixed sequence of forget requests, our method keeps held-out utility (MUSE utility, MMLU) significantly higher than both SimNPO+RT and first-order projection, measured by paired bootstrap over trials at matched forget effectiveness. It must do so without lowering Forget Quality below SimNPO's level on single requests.

Why not just…?

Why not just add a retain loss? A retain penalty trades keep loss against forget loss through one weight and has no mechanism that cancels drift. SciUnlearn shows it stops the MMLU collapse but leaves paraphrase suppression untouched. MUSE shows soft regularization does not prevent sequential degradation, and UEPI's Hénon-Heiles comparison shows the same soft-versus-hard gap in physics. The sequential experiment tests this directly.

Why not just project out the retain gradient? This is the real threat, and it is the ablation built into the design. If one-signed drift is small relative to other error sources in practice, first-order projection will match us at lower cost, and we will conclude the discrete gradient is unnecessary. The measured second-order drift term and the sequential curves settle it.

Why not just take smaller steps? With total parameter displacement fixed, the $n h^2$ drift scales as displacement times $h$, so shrinking steps does reduce drift linearly. The honest comparison is at matched compute. Our method wins only if it reaches a given drift level more cheaply than simply taking more, smaller steps. We report the full drift-versus-compute frontier for both.

Why not just use SimNPO on more data? The coreset result shows that larger forget sets are mostly redundant. They do not address single-record requests, which is the regime where drift from many small sequential requests accumulates.

Proposed Experiments

Sequential single-record unlearning. We run MUSE-News and MUSE-Books sustainability sequences, plus a TOFU protocol built as a chain of single-author requests, 20 QA pairs each, up to dozens of consecutive requests. We plot anchor-loss drift, held-out utility, KnowMem, and membership-inference leakage against request index. The decisive check is whether first-order projection shows a one-directional rise in keep loss that grows with request count, and whether the discrete-gradient step flattens it.

A cohort of single cases. To evaluate as a cohort rather than an anecdote, we treat each single TOFU author and each single SciUnlearn claim as one "patient." We run at least 50 independent trials per method with shared seeds for paired comparison. We report median and tail outcomes for Forget Quality, target accuracy ($F_1$), and paraphrase accuracy ($F_2$) on OLMo-3-7B-Instruct, the model used for SciUnlearn.

Paraphrase honesty check. Conserving the keep loss might block forgetting of paraphrases, if they share parameters with retained knowledge. We check whether our method's $F_2$ drop is no worse than SimNPO's. A worse $F_2$ drop would show that conservation buys utility by restricting forgetting, and we would report it as such.

Anchor generalization. We check whether conserving the anchor losses predicts conservation of held-out utility. If held-out utility drifts while the anchors stay fixed, the method is optimizing a proxy, and anchor design becomes the main open question.