Is your algorithm unlearning or untraining?

Eleni Triantafillou, Ahmed Imtiaz Humayun, Monica Ribero, Alexander Matt Turner, Michael C. Mozer, Georgios KaissisarXiv 2026 · arXiv

TL;DR

This paper clarifies a major point of confusion in the machine learning literature by splitting the overloaded term "machine unlearning" into two distinct concepts: Untraining (removing the influence of specific training samples) and Unlearning (erasing an entire underlying concept or behavior). Recognizing this distinction is crucial for choosing the right evaluation metrics, setting proper baselines, and designing algorithms that actually achieve safety or privacy goals.

What Problem Does This Paper Address?

As machine learning models grow larger and ingest massive datasets, the need to post-hoc "delete" specific data points or behaviors has skyrocketed. Currently, the term "machine unlearning" is used as a catch-all for everything from honoring a user's "right to be forgotten" (privacy) to removing toxic behaviors or copyrighted content from large language models (safety).

The authors argue that this overloading of terminology creates massive confusion. Researchers are applying the same algorithms and evaluation metrics to fundamentally different problems. This leads to mismatched expectations, flawed baselines, and misinterpretations of whether an algorithm has actually succeeded.

Methods

As a conceptual position paper, the authors do not introduce a new empirical algorithm. Instead, they formalize a rigorous framework to separate the two paradigms mathematically and conceptually:

  • Untraining: Reverses empirical risk minimization on a specific "forget set" $S$. The ideal baseline is simply retraining the model from scratch on the remaining data ($D \setminus S$). It aims to match this distribution exactly. Crucially, if the retrained model would still correctly predict an item due to normal generalization, the untrained model should too.
  • Unlearning: Aims to remove the entire underlying distribution, concept, or behavior $B$ represented by a forget set $S$. The authors define this by introducing $S_{full}$—the complete, hypothetical set of all possible examples of that behavior. The ideal baseline here is a model trained on $D \setminus S_{full}$. Unlearning requires the algorithm to generalize the removal beyond the specific examples provided in $S$.

Datasets

Not discussed in the paper. (As a conceptual note, the paper does not introduce new datasets or perform empirical experiments, though it references existing benchmarks like MUSE and WMDP to categorize prior work).

Key Insights and Ideas

  • Untraining does not imply misclassification: If you untrain a model on a few specific examples of a common concept (e.g., images of circles), a perfectly untrained model should still classify them correctly if there are plenty of other circles left in the training set to generalize from.
  • Visualizing the Split: The core difference is elegantly captured in Figure 1:
Illustration of Untraining vs. Unlearning

As shown above, Untraining only removes the specific influence of the forget set $S$ (the star becomes incorrectly predicted because stars are rare, but circles remain correct). Unlearning removes the entire concept of "being a circle or a star," causing the model to fail on all instances of that behavior.

  • Metric Mismatch: Traditional metrics like Membership Inference Attacks (MIAs) are great for Untraining (verifying privacy) but completely inappropriate for Unlearning (verifying safety). A model could pass an MIA (proving specific data points were removed) while still retaining full, dangerous knowledge of the broader concept.

Key Results

The paper provides a valuable taxonomy mapping existing literature to the correct definition:

  • Mapped to Untraining: User privacy requests (GDPR compliance), removing mislabeled/noisy data points, and preventing the verbatim extraction of specific copyrighted sentences.
  • Mapped to Unlearning: Erasing dangerous knowledge (e.g., bioweapons), removing "Not Safe For Work" (NSFW) concepts, eliminating backdoors/poisoning, and erasing artistic styles.

Limitations and Future Work

  • Scoping Concepts: The proposed definition of Unlearning relies on concepts that can be explicitly defined by a set of examples $S$. Abstract capabilities that cannot be easily isolated this way (e.g., "edge detection") fall outside the scope of this framework.
  • Open Questions: The authors highlight several critical, unexplored research directions: How large does the forget set $S$ need to be relative to $S_{full}$ for unlearning to succeed? How can we design algorithms with the right inductive biases to generalize concept removal from just a few examples ("few-shot unlearning")?

Summary updated

Abstract

As models are getting larger and are trained on increasing amounts of data, there has been an explosion of interest into how we can ``delete'' specific data points or behaviours from a trained model, after the fact. This goal has been referred to as ``machine unlearning''. In this note, we argue that the term ``unlearning'' has been overloaded, with different research efforts spanning two distinct problem formulations, but without that distinction having been observed or acknowledged in the literature. This causes various issues, including ambiguity around when an algorithm is expected to work, use of inappropriate metrics and baselines when comparing different algorithms to one another, difficulty in interpreting results, as well as missed opportunities for pursuing critical research directions. In this note, we address this issue by establishing a fundamental distinction between two notions that we identify as \unlearning and \untraining, illustrated in Figure 1. In short, \untraining aims to reverse the effect of having trained on a given forget set, i.e. to remove the influence that that specific forget set examples had on the model during training. On the other hand, the goal of \unlearning is not just to remove the influence of those given examples, but to use those examples for the purpose of more broadly removing the entire underlying distribution from which those examples were sampled (e.g. the concept or behaviour that those examples represent). We discuss technical definitions of these problems and map problem settings studied in the literature to each. We hope to initiate discussions on disambiguating technical definitions and identify a set of overlooked research questions, as we believe that this a key missing step for accelerating progress in the field of ``unlearning''.
BibTeX
@misc{triantafillou2026is,
  author = {Eleni Triantafillou and Ahmed Imtiaz Humayun and Monica Ribero and Alexander Matt Turner and Michael C. Mozer and Georgios Kaissis},
  title = {{Is your algorithm unlearning or untraining?}},
  year = {2026},
  eprint = {2604.07962},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2604.07962},
}
Figure