Instance Level Machine Unlearning

48 concepts43 papers

When a user invokes the "right to be forgotten" under modern privacy regulations, they rarely ask to delete an entire category of knowledge. Instead, they ask to erase a specific piece of personal data—a single photo, a private address, or a specific text interaction.

Traditional machine unlearning methods were built for coarse-grained removal, such as deleting an entire semantic class (Boundary Unlearning: Rapid Forgetting of Deep Networks via Shifting the Decision Boundary) or an entire task subspace (UNLEARN Efficient Removal of Knowledge in Large Language Models). However, shifting from category-level erasure to instance-level machine unlearning introduces a fundamental technical challenge: how do you surgically remove a single data point when its representations are deeply intertwined with the data you want to keep?

The Challenge of Feature Entanglement

To understand why instance-level unlearning is difficult, consider how deep neural networks store information. In class-level unlearning, a model can suppress an entire activation space or shift a macro decision boundary because the target data forms a distinct semantic cluster.

But an individual sample shares a nearly identical feature distribution with the remaining data in its parent class (Deep Unlearning: Fast and Efficient Training-free Approach to Controlled Forgetting). As a result, the target instance activates many of the same internal features and sparse keys as the data that must be retained (Low Compute Unlearning via Sparse Representations).

Class-Level Erasure (Coarse)         Instance-Level Erasure (Fine-Grained)
       [ Data Class ]                       [ Data Class ]
      /   /   |   \   \                    /   /   |   \   \
     X   X    X    X   X                  ·   ·   [X]   ·   ·
  (Delete entire category)              (Surgically isolate one sample)

If an unlearning algorithm aggressively masks these shared representations, it causes over-unlearning—collateral damage that degrades the model's performance on permissible, highly similar content (Unlearning’s Blind Spots: Over‑Unlearning and Prototypical Relearning Attack; Fine-Grained Erasure in Text-to-Image Diffusion-based Foundation Models). The goal of instance-level unlearning is to achieve high-precision erasure without breaking these shared semantic structures.

Surgical Subspace and Boundary Calibration

To bypass feature entanglement, researchers have developed mechanisms to isolate the exact mathematical footprint of a single instance.

One approach operates within the model's weight and feature spaces. Instead of updating parameters globally, algorithms can confine changes to regions that do not impact the rest of the dataset. For example, Xin Yu introduces Machine Unlearning via Null Space Calibration, a method that protects remaining knowledge by restricting weight updates entirely to a mathematical null space. At the feature level, Tae-Young Lee et al. propose ESC: Erasing Space Concept for Knowledge Deletion, which uses Singular Value Decomposition (SVD) to identify and prune highly specific internal memory spaces rather than altering the final classification layers.

Another strategy modifies the local error surface rather than the global architecture. Instead of shifting a macro decision boundary, the model can smooth out the boundary locally around the target sample. Kyle Fogarty et al. demonstrate this in An Information Theoretic Approach to Machine Unlearning via "Just in Time" (JiT) unlearning. By minimizing local gradients specifically around the data points to be forgotten, the method flattens the model's response to those instances without requiring access to the original training set.

The Heterogeneity of Forgetting

A key insight driving instance-level research is that not all data points are equally easy to forget. In category-level unlearning, a class's proximity to a decision boundary often dictates how easily it can be removed. However, Hassan Sajjad et al. show that this rule breaks down at the sample level; individual data points exhibit highly variable resistance to forgetting based on how deeply they were memorized during training (Towards Understanding the Feasibility of Machine Unlearning).

To quantify this phenomenon, XiaoHua Feng et al. introduce the Memory Removal Difficulty (MRD) metric in A Neuro-inspired Interpretation of Unlearning in Large Language Models through Sample-level Unlearning Difficulty. By measuring sample-level difficulty, unlearning frameworks can dynamically allocate optimization focus. For instance, Xiang Li et al. leverage this heterogeneity in LoReUn: Data Itself Implicitly Provides Cues to Improve Machine Unlearning by reweighting target samples based on their evaluation loss, prioritizing updates on the specific instances the model has memorized most stubbornly.

Native Mechanisms for Granular Erasure

Because post-hoc optimization of entangled weights is inherently complex, an alternative paradigm seeks to build instance-level unlearning directly into the model architecture from day one.

Rutger Hendrix proposes this shift in Pre-Forgettable Models: Prompt Learning as a Native Mechanism for Unlearning. Instead of allowing data points to alter a shared parameter backbone during training, specific knowledge is bound to individual, learnable prompt tokens while the core model remains frozen. When a privacy request arrives, instance-level unlearning becomes trivial and mathematically absolute: the system simply deletes the corresponding token. This architectural isolation completely avoids the risk of collateral damage, providing a clean solution to the problem of feature entanglement.

Key Foundational Work

Boundary Unlearning: Rapid Forgetting of Deep Networks via Shifting the Decision Boundary (2023) — contributed 2 concept elements to this direction.
Federated Unlearning via Class-Discriminative Pruning (2022) — contributed 1 concept elements to this direction.
Unlearning via Sparse Representations (2023) — contributed 1 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_unlearning_granularity_4bf7910f,
  title = {unlearning granularity},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/unlearning-granularity/rdr_4bf7910f01814688aa3568c6b16cf6a1},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_unlearning_granularity_4bf7910f_v1,
  title = {unlearning granularity},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/unlearning-granularity/rdr_4bf7910f01814688aa3568c6b16cf6a1/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}

See something wrong? Report a mistake