Exact Unlearning of Finetuning Data via Model Merging at Scale

Kevin Kuo, Amrith Setlur, Kartik Srinivas, Aditi Raghunathan, Virginia Smith

TL;DR

SIFT-Masks (SIgn-Fixed Tuning-Masks) enables exact machine unlearning at scale by combining model merging with a novel "sign-fixed" finetuning approach. It achieves accuracy comparable to centralized training while being up to 250x more computationally efficient for unlearning requests, even when scaling to hundreds of distinct tasks.

What Problem Does This Paper Address?

As AI models become more integrated into daily life, the "right to be forgotten" (mandated by regulations like GDPR) has become a technical nightmare. If a user asks to have their data removed, a model provider must "unlearn" that data.

Current approaches fall into two camps, both flawed:

  1. Approximate Unlearning: These methods (like gradient ascent on the data to be forgotten) are fast but "leaky." They don't provide guarantees, and clever attacks can often recover the supposedly deleted information.
  2. Exact Unlearning: The gold standard is retraining the model from scratch without the target data. However, for Large Language Models (LLMs), this is prohibitively expensive and slow, especially if deletion requests arrive frequently.

Model merging offers a potential middle ground: finetune separate models on separate data shards and average them. To unlearn, you simply subtract the relevant "task vector." The problem? Simple averaging destroys model performance as you scale to hundreds of tasks, and sophisticated merging methods that fix this performance drop usually require "global" information that makes exact unlearning expensive again.

Methods

The authors introduce SIFT-Masks, a framework designed to maintain the efficiency of model merging while recovering the performance lost during the merge.

Figure 2: SIFT-Masks starts with (1) Sign-Fixed (Fine)-Tuning, which initializes a random sign vector v and constrains finetuning to match this sign vector or otherwise be sparse, producing a sparse local model Mi with mask mi. We then (2) merge these local models into a global model and only keep the masks mi. When (3) serving task ci, we apply mi to the merged model. Finally, to unlearn task ci, we simply unmerge Mi from M and discard mi.

The core innovation is a two-step process:

  • Sign-Fixed Tuning (SIFT): Before any training begins, a global random sign vector $v$ is generated. During local finetuning for a specific task, the model's weight updates are constrained to match the signs in $v$. If an update would flip a sign, it is clipped to zero. This creates a sparse, task-specific mask $m_i$ that is independent of all other tasks.
  • Localization: Instead of just using a single merged model for everyone, the system stores the merged model and the small, sparse masks for each task. When serving a specific task, the mask is applied to the merged task vector.

Because the masks are determined independently during training (thanks to the fixed sign vector), unlearning remains a simple subtraction: $Merged\_Model_{new} = Merged\_Model_{old} - Task\_Vector_{target}$.

Datasets

The researchers evaluated SIFT-Masks across four diverse text datasets, scaling up to 500 individual tasks (clients/users):

  • TOFU: A benchmark specifically for fictitious unlearning in LLMs.
  • Sent140: Sentiment analysis on Twitter data, partitioned by user.
  • Reddit: Subreddit classification, partitioned by user.
  • StackOverflow: Programming tag prediction, partitioned by user.

They tested these using state-of-the-art open models, including Llama 3.2-1B, GPT2-XL (1.5B), and FLAN-T5-Large (700M).

Key Insights and Ideas

  • Independence is the Secret Sauce: Previous high-performance merging methods (like TIES-merging or TALL-masks) calculate masks or sign agreements after seeing all models. This creates a dependency that forces you to re-calculate everything when one model is removed. By fixing the signs before training, SIFT-Masks breaks this dependency.
  • The Scaling Wall: The paper highlights that naive model merging (simple averaging) hits a performance wall very quickly. As shown in Figure 3, while merging two models might work fine, merging 200 models can degrade performance to near-zero (zero-shot levels).
  • Task Heterogeneity Matters: Merging is most effective when tasks are distinct. If tasks have conflicting data, a single central model struggles to learn them all simultaneously, whereas merged local models with masks can preserve these nuances.

Key Results

SIFT-Masks effectively bridges the gap between the high accuracy of expensive retraining and the high efficiency of naive merging.

Figure 1: SIFT-Masks is a merging-and- localization method that can match cen- tral training accuracy while being as effi- cient for unlearning as na¨ıve merging.
  • Accuracy Recovery: SIFT-Masks improves accuracy by 5-80% over naive merging. On the TOFU dataset, while naive merging dropped to 10% accuracy at scale, SIFT-Masks maintained 99% accuracy.
  • Massive Efficiency Gains: Compared to retraining a central model (the only other way to get "exact" unlearning), SIFT-Masks is up to 250x cheaper in terms of compute.
  • Scalability: The method was successfully tested merging up to 500 models, an order of magnitude more than most prior model merging research.
Figure 3: Left: Merging (x > 1) degrades performance (answer probability) compared to applying local models (x = 1); this issue becomes more severe as the number of models increases, potentially reducing performance to zeroshot accuracy. Right: Our method SIFT-Masks recovers performance (probability for TOFU; accuracy otherwise) from merging and suffers much less from scale.

Limitations and Future Work

  • Storage Overhead: While compute is saved, storage is increased. The system must store the base model, the merged task vector, and a sparse mask for every single task. The authors note these masks are small (1/32 the size of a full model), but for millions of users, this could still be significant.
  • The "Central" Gap: In some settings (like StackOverflow), a centrally trained model still outperforms merged models because it can leverage similarities across tasks more effectively. Future work could explore ways to better capture these global features without sacrificing unlearning efficiency.
  • Base Model Dependency: The success of merging relies heavily on having a strong, pre-trained base model. The authors suggest that future research should investigate how the quality of the base model impacts the limits of unlearning at scale.

Summary updated

Abstract

Approximate unlearning has gained popularity as an approach to efficiently update an LLM so that it behaves (roughly) as if it was not trained on a subset of data to begin with. However, existing methods are brittle in practice and can easily be attacked to reveal supposedly unlearned information. To alleviate issues with approximate unlearning, we instead propose SIFT-Masks (SIgn-Fixed Tuning-Masks), an exact unlearning method based on model merging. SIFT-Masks addresses two key limitations of standard model merging: (1) merging a large number of tasks can severely harm utility; and (2) methods that boost utility by sharing extra information across tasks make exact unlearning prohibitively expensive. SIFT-Masks solves these issues by (1) applying local masks to recover task-specific performance; and (2) constraining finetuning to align with a global sign vector as a lightweight approach to determine masks independently before merging. Across four settings where we merge up to 500 models, SIFT-Masks improves accuracy by 5-80% over naive merging and uses up to 250x less compute for exact unlearning compared to other merging baselines.
BibTeX
@inproceedings{kuo2025exact,
  author = {Kevin Kuo and Amrith Setlur and Kartik Srinivas and Aditi Raghunathan and Virginia Smith},
  title = {{Exact Unlearning of Finetuning Data via Model Merging at Scale}},
  booktitle = {MCDC @ ICLR},
  year = {2025},
  url = {https://openreview.net/forum?id=u89LDBIyDe},
}
Figure