Temper-Then-Tilt: Principled Unlearning for Generative Models through Tempering and Classifier Guidance

Jacob. Block, Mehryar Mohri, Aryan Mokhtari, Sanjay Shakkottai

Abstract

We study machine unlearning in large generative models by framing the task as density ratio estimation to a target distribution rather than supervised fine-tuning. While classifier guidance is a standard approach for approximating this ratio and can succeed in general, we show it can fail to faithfully unlearn with finite samples when the forget set represents a sharp, concentrated data distribution. To address this, we introduce Temper-Then-Tilt Unlearning (T3-Unlearning), which freezes the base model and applies a two-step inference procedure: (i) tempering the base distribution to flatten high-confidence spikes, and (ii) tilting the tempered distribution using a lightweight classifier trained to distinguish retain from forget samples. Our theoretical analysis provides finite-sample guarantees linking the surrogate classifier's risk to unlearning error, proving that tempering is necessary to successfully unlearn for concentrated distributions. Empirical evaluations on the TOFU benchmark show that T3-Unlearning improves forget quality and generative utility over existing baselines, while training only a fraction of the parameters with a minimal runtime.

Introduction

Large generative models often memorize sensitive, copyrighted, or private information. Machine unlearning aims to remove the influence of specific "forget sets" from a pre-trained model while keeping the rest of its knowledge (the "retain set") intact.

The standard approach is supervised fine-tuning (SFT), but this is computationally expensive and can cause "catastrophic forgetting" of unrelated skills. An alternative is classifier guidance, which uses a density ratio to reweight the model's outputs. However, the authors identify a critical failure mode: when the data to be forgotten is highly specific or "sharp" (concentrated in a small region of the probability space), even tiny errors in the classifier lead to massive information leakage. Standard methods simply cannot "tilt" the distribution hard enough to suppress these sharp peaks with finite training samples.

Methods: The T3-Unlearning Framework

The authors propose T3-Unlearning, a two-stage inference-time procedure that keeps the base model frozen and shifts the perspective from parameter updates to distribution correction.

The T3-Unlearning architecture: A lightweight classifier is trained on frozen hidden states, and inference combines tempering with classifier-based tilting.

1. Classifier Training

A lightweight probabilistic classifier (a simple linear head) is trained on top of the frozen hidden states of the base model. This classifier learns to distinguish between samples from the "retain" set and the "forget" set. Because the base model is frozen and hidden states can be cached, the training process is essentially just a quick linear regression over pre-computed features.

2. Tempered Inference

Instead of applying the classifier directly to the base model, T3-Unlearning performs two steps during sampling:

  • Tempering: A temperature $T > 1$ is applied to the base distribution ($p^{1/T}$). This "flattens" the distribution, smoothing out the sharp, high-confidence spikes that are difficult to unlearn.
  • Tilting: The flattened distribution is then "tilted" using the learned classifier to suppress forget-set tokens.

The core innovation is the tempering step. Tempering acts as a safety buffer; by making the model "less sure" about everything globally, the classifier can more effectively suppress the specific things it should not generate. Theoretical analysis shows that tempering is mathematically necessary to bound the "Forget Error" when dealing with concentrated distributions.

Experimental Evaluation

The framework was evaluated on the TOFU (Task of Fictitious Unlearning) benchmark, consisting of a Llama-based model fine-tuned on fictitious author biographies, and a synthetic Gaussian benchmark to validate theoretical claims.

Key Results

  • Superior Forget Quality: On the TOFU benchmark, T3-Unlearning achieved a Forget Quality (FQ) of 0.914 (on the 5% split), significantly outperforming popular baselines like NPO (0.793) and SimNPO (0.745).
  • Preserved Utility: T3 maintained high generative utility (0.900 MU-ROUGE), whereas many other high-forgetting methods saw their utility drop significantly.
  • Massive Speedup: In "preprocessed" mode, T3-Unlearning takes only ~5 to 7 seconds to run on the TOFU dataset. For comparison, standard fine-tuning baselines take between 200 and 1,200 seconds.
  • Parameter Efficiency: T3 trains only 2.65 million parameters (a tiny linear head), which is roughly 0.03% of the total parameters in a Llama 3.1 8B model.

Conclusion and Future Work

T3-Unlearning provides a principled and highly efficient method for machine unlearning. By combining tempering with classifier guidance, it overcomes the "sharpness trap" that plagues standard density ratio estimation methods. While tempering introduces a global bias that must be balanced against unlearning quality, the method offers a significant improvement in both speed and reliability. Future research may extend this framework to other generative modalities, such as diffusion models for image generation.

Summary updated

Abstract

We study machine unlearning in large generative models by framing the task as density ratio estimation to a target distribution rather than supervised fine-tuning. While classifier guidance is a standard approach for approximating this ratio and can succeed in general, we show it can fail to faithfully unlearn with finite samples when the forget set represents a sharp, concentrated data distribution. To address this, we introduce Temper-Then-Tilt Unlearning (T3-Unlearning), which freezes the base model and applies a two-step inference procedure: (i) tempering the base distribution to flatten high-confidence spikes, and (ii) tilting the tempered distribution using a lightweight classifier trained to distinguish retain from forget samples. Our theoretical analysis provides finite-sample guarantees linking the surrogate classifier's risk to unlearning error, proving that tempering is necessary to successfully unlearn for concentrated distributions. Empirical evaluations on the TOFU benchmark show that T3-Unlearning improves forget quality and generative utility over existing baselines, while training only a fraction of the parameters with a minimal runtime.
BibTeX
@inproceedings{block2026temper,
  author = {Jacob. Block and Mehryar Mohri and Aryan Mokhtari and Sanjay Shakkottai},
  title = {{Temper-Then-Tilt: Principled Unlearning for Generative Models through Tempering and Classifier Guidance}},
  booktitle = {International Conference on Machine Learning},
  year = {2026},
  url = {https://openreview.net/forum?id=7Oc4ARFCTV},
}
Figure