Robust Verification of Data Erasure in Machine Unlearning

This direction is split into other directions: membership inference vulnerability (over-forgetting privacy risk), unlearning-induced privacy leakage. Its last version stays readable and citable here.

Version 1 of 1294 papers546 conceptsfrozen 2026-10-04

Machine unlearning (MU) is often framed as a way to satisfy the "right to be forgotten" or to purge poisoned data without the prohibitive cost of retraining a model from scratch. However, a growing body of evidence suggests that many unlearning techniques create an illusion of forgetting. While a model might stop predicting a sensitive label as its top choice, the underlying information often remains embedded in its weights, latent representations, and probability distributions.

To achieve true unlearning, researchers are moving beyond simple accuracy metrics toward robust verification methods that treat the model not as a deterministic function, but as a complex statistical distribution.

The Failure of Top-1 Accuracy

The most intuitive way to check if a model has "forgotten" a data point is to see if it still gets the answer right. If the accuracy on the "forget set" drops to zero, we might assume the data is gone.

Wee Peng Tay and Yahya Alkhatib argue in Conformal Machine Unlearning that this is a "fake unlearning" problem. A model can be forced to output the wrong label while still assigning a high relative probability to the correct one. If a prediction set—a range of likely labels—still contains the deleted data's true label, the model's internal logic hasn't actually purged the information.

This persistence is even more pronounced in Large Language Models (LLMs). Hadi Reisizadeh and colleagues introduced the leak@k metric in Leak@k: Unlearning Does Not Make LLMs Forget Under Probabilistic Decoding to show that sensitive data reliably resurfaces under probabilistic sampling. Even if the "greedy" (most likely) token is changed, the original sensitive information often hides in the lower-probability "tail" of the distribution, waiting to be sampled.

Residuals in the Latent Space

Even if the output layer is successfully scrubbed, the model’s internal "features" may still retain structural remnants of the data. This is what Anjie Le and co-authors address in POUR: A Provably Optimal Method for Unlearning Representations via Neural Collapse. They find that simple output masking fails because the model's internal feature space remains influenced by the original samples.

This phenomenon isn't limited to classification. In 3D scene representation, Simona Kocour demonstrates in Is there anything left? Measuring semantic residuals of objects removed from 3D Gaussian Splatting that "deleting" an object often just makes it invisible to humans. Machine learning models can still detect "semantic residuals"—subtle data traces left behind in the scene's underlying geometry.

Furthermore, Hsiang Hsu and Pradeep Niroula show in The Unseen Threat: Residual Knowledge in Machine Unlearning under Perturbed Samples that unlearning often only works for the exact data point. If the forgotten sample is slightly perturbed with noise, the model may suddenly "remember" it, proving that the influence of the data persists in the local neighborhood of the sample.

The Scar of the Unlearning Process

A paradox of machine unlearning is that the act of removing data can itself leave a "scar"—a detectable behavioral signature that the data was once there.

Zhengbao He and Zhehao Huang describe this as "over-forgetting" in Towards Natural Machine Unlearning. When a model is fine-tuned to forget, it often produces unnaturally neutral or nonsensical outputs for those specific points. This creates an "entropy gap" between forgotten data and data the model truly never saw.

Sungjun Cho and Dasol Hwang formalize this critique with the FADE metric in Reference-Specific Unlearning Metrics Can Hide the Truth. They argue that most unlearning methods fail to achieve "functional alignment" with the gold standard: a model retrained from scratch without the target data. Instead, the unlearned model becomes a "noisy labeler" that leaks privacy.

Verifying Erasure via Adversarial Inference

If unlearning is successful, an attacker should not be able to distinguish between a model that has undergone unlearning and one that was never trained on the sensitive data in the first place.

Membership Inference Attacks (MIAs) have become the primary tool for this verification. James Joshi and Liou Tang developed Apollo, a label-only attack that can identify deleted data just by observing shifts in the model's decision boundaries. Similarly, Liang Hu and Zhihao Sui demonstrate in Recalling The Forgotten Class Memberships that even when accuracy is zeroed out, an external "student" model can be trained to recover the forgotten class memberships by treating the unlearned model as a noisy teacher.

Toward Robust Solutions

To move past these vulnerabilities, the field is shifting toward more rigorous frameworks:

  1. Conformal Prediction: Frameworks like EFFACE (proposed by Wee Peng Tay) and work by Ren Wang use uncertainty sets to provide statistical guarantees that the true label is no longer prioritized by the model's internal logic.
  2. Epistemic Uncertainty: Thomas Liebig suggests using the Fisher Information Matrix to measure how much information a model retains, providing a theoretically grounded "efficacy score" for unlearning.
  3. Exact Unlearning Architectures: Systems like ARCANE, developed by Haonan Yan and Hui Li, use ensemble-based methods to allow for the complete removal of data influence, bypassing the "residual" problems of approximate fine-tuning.

The consensus across these 205 papers is clear: unlearning is not a matter of changing a model's answer, but of erasing its statistical memory. Until a model is indistinguishable from one that never saw the data, the "right to be forgotten" remains unfulfilled.

Papers in this version

Showing 60 of 294 papers.

History & citationVersion 1 · 2026-10-04

Split into other directions · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_robust_verification_of_data_erasure_in_m_a4ae361e,
  title = {Robust Verification of Data Erasure in Machine Unlearning},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/robust-verification-of-data-erasure-in-machine-unlearning/rdr_a4ae361e6e8542cbb0462c7562b94991},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-19},
}

This exact version (version 1, frozen text):

@misc{lacuna_robust_verification_of_data_erasure_in_m_a4ae361e_v1,
  title = {Robust Verification of Data Erasure in Machine Unlearning},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/robust-verification-of-data-erasure-in-machine-unlearning/rdr_a4ae361e6e8542cbb0462c7562b94991/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-19},
}