Quantization-Robust Unlearning through the Lens of Retain-Forget Loss Landscapes Interaction

Jialu Wang, Jianing Deng, Shuqing Luo, Yuanzhe Li, Dongwei Wang, Jingtong Hu, Huanrui Yang, Song Wang, Tianlong Chen

TL;DR

Machine unlearning successfully removes sensitive or copyrighted data from Large Language Models (LLMs), but subsequent post-training compression like 4-bit quantization often causes the model to "re-learn" or recover this deleted data. This paper introduces a quantization-robust unlearning framework that smooths out the sharp forget-loss landscape using sensitivity-guided noisy regularization and restricts parameter updates to forget-critical layers. The result is an LLM that successfully maintains its unlearning behavior even after aggressive 4-bit quantization, without sacrificing overall model utility.

What Problem Does This Paper Address?

To comply with privacy laws (like GDPR) and copyright restrictions, AI practitioners use machine unlearning to erase the influence of specific training data subsets from an LLM without the prohibitive cost of retraining the model from scratch.

However, real-world deployment requires efficiency. LLMs are typically compressed using low-bit quantization (e.g., converting 32-bit floating-point weights to 4-bit integers) to fit on commercial hardware. The authors highlight a critical vulnerability in this pipeline: catastrophic unlearning failure under quantization. An LLM that appears to have successfully forgotten copyrighted data in full precision will suddenly recover that knowledge once it is quantized to 4 bits.

Figure 1: Catastrophic failure of LLM unlearn- ing under quantization. Gap between full pre- cision and 4-bit quantization on MUSE.

This happens because the rounding errors introduced by quantization act as parameter perturbations, shifting the model out of its fragile "forgotten" state and restoring the supposedly deleted data.

Methods

The authors propose a two-pronged framework to make unlearning resilient to downstream quantization:

Overview of the quantization-robust unlearning framework
  1. Sensitivity-Guided Noisy Regularization: The authors analyze the loss landscape and find that standard unlearning forces the model into a very narrow, sharp basin for the forget loss. To fix this, they use the diagonal empirical Fisher Information Matrix as a tractable proxy for Hessian curvature to identify weights highly sensitive to the ratio of forget-to-retain loss. During the unlearning process, they inject rounding-based noise into the top-$K$ most sensitive weight channels. This noise mimics downstream quantization errors, forcing the optimization algorithm to steer toward a much flatter minimum.

  2. Forget-Critical Optimization: Updating every weight in a massive LLM during unlearning causes collateral damage to its general knowledge base (the retain set). The authors compute first-order gradient norms to score each layer's impact on forgetting versus retaining. They restrict gradient updates exclusively to the top-scoring "forget-critical" layers while freezing the rest of the network, preserving the model's core utility.

Datasets

The framework was validated across multiple unlearning algorithms on two prominent benchmarks:

  • MUSE (Machine Unlearning Six-Way Evaluation): Evaluated using MUSE-BOOKS (removing Harry Potter books from an ICLM-7B model) and MUSE-NEWS (removing BBC News articles from a LLaMA2-7B model).
  • TOFU (Task of Fictitious Unlearning): A synthetic benchmark consisting of fictional autobiographical question-answering data, evaluated on LLaMA3-1B and LLaMA3-8B across various forget fractions (1%, 5%, and 10%).

Key Insights and Ideas

  • The Sharpness Discrepancy: The root cause of quantization-induced memory recovery is that the forget-loss landscape converges to a sharp basin, whereas the retain-loss landscape is naturally much flatter. Quantization rounding easily knocks the model out of a sharp forget basin, but leaves it safely within the flat retain basin.
  • A Tiny Perturbation Goes a Long Way: You do not need to regularize the entire model to achieve robustness. The authors found that perturbing just a tiny fraction (~0.1%) of the most sensitive parameters during unlearning is enough to smooth out the landscape.
Comparison of forget-loss landscapes before and after regularization
  • Layer Specialization Prevents Interference: Not all layers contribute equally to unlearning. By isolating and updating only the layers highly responsible for the forget set, the model avoids a poor forget-utility trade-off.

Key Results

  • Suppressed Knowledge Recovery: On MUSE-NEWS, standard unlearning methods saw forgotten knowledge metrics jump from ~13% in full precision to ~54% after 4-bit quantization. The proposed method successfully suppressed this recovery, outperforming the prior state-of-the-art method (SURE) by up to 8.0% in the retain-forget gap.
  • Maintained Utility: While aggressive unlearning often causes general model performance to collapse after quantization, this framework kept the quantized model's utility closely aligned with its full-precision counterpart.
  • Quantization Agnostic: The method demonstrated robust generalization across different post-training quantization techniques, including standard bitsandbytes, AWQ, and GPTQ.

Limitations and Future Work

  • Curvature Proxy Dependence: The method relies on diagonal empirical Fisher information as a proxy for the Hessian because computing the full Hessian for LLMs is computationally intractable. While effective, this remains an approximation.
  • Hyperparameter Tuning: The framework requires tuning the parameter selection ratio ($\tau$) and determining the optimal number of forget-critical layers ($K$), which can vary depending on the model architecture and the specific unlearning task.
  • Focus on PTQ: The study primarily focuses on Post-Training Quantization (PTQ) scenarios. Exploring how these insights apply to Quantization-Aware Training (QAT) during unlearning remains an open direction for future research.

Summary updated

Abstract

Unlearning ensures LLM compliance by removing the influence of private or copyrighted training data. However, since LLM models typically undergo post-training compression, like quantization, in practical deployment, it has been observed that the unlearning effect can be substantially weakened, with the forgetting behavior degrading more severely than that of model utility. This paper proposes a quantization-robust unlearning framework that makes forgetting robust to quantization while maintaining overall model utility. We analyze this gap through the lens of loss landscape. Specifically, our analysis reveals a curvature-based criteria that pinpoints sensitive weights in the unlearned model that leads to both non-robust forgetting and reduced utility. We therefore propose sensitivity-guided noisy regularization, which is applied on the sensitive parameters to steer the model convergence towards a smoother minima of uniformly low forget and retain losses. Balancing unlearning and utility, we further propose forget-critical optimization, which updates only forget-critical layers, preserving most of the network to retain useful knowledge. Extensive experiments on the MUSE and TOFU benchmarks across multiple LLM unlearning algorithms show that our approach achieves substantially more quantization-resilient forgetting while maintaining utility.
BibTeX
@inproceedings{wang2026quantization,
  author = {Jialu Wang and Jianing Deng and Shuqing Luo and Yuanzhe Li and Dongwei Wang and Jingtong Hu and Huanrui Yang and Song Wang and Tianlong Chen},
  title = {{Quantization-Robust Unlearning through the Lens of Retain-Forget Loss Landscapes Interaction}},
  booktitle = {COLM},
  year = {2026},
  url = {https://openreview.net/forum?id=WZecHZcKN9},
}
Figure