Are we truly forgetting? A critical re-examination of machine unlearning evaluation protocols

Yongwoo Kim, Sungmin Cha, Donghyun Kim

TL;DR

Existing machine unlearning evaluations often provide a false sense of security by focusing only on output logits, which models can "game" through shortcut learning. This paper introduces a more rigorous benchmark using representation-based metrics and a "Top Class-wise Forgetting" scenario to reveal that many state-of-the-art methods fail to actually erase sensitive information from a model's internal feature space.

What Problem Does This Paper Address?

Machine unlearning is the process of removing the influence of specific data points from a trained model—essential for complying with "the right to be forgotten" (like GDPR) or removing biased data. Ideally, an unlearned model should behave exactly like a model that was never shown that data in the first place (the "retrained" gold standard).

The authors identify a critical flaw in how we measure success: current protocols rely almost exclusively on logit-based metrics (like accuracy on the "forget set"). They argue that this is narrow and misleading. A model can easily change its final classification head to "hide" certain classes while leaving the underlying feature representations (the encoder) completely intact. If the encoder still "recognizes" the forbidden data, the unlearning is superficial and potentially reversible.

Methods

The researchers propose a holistic evaluation framework that shifts the focus from what a model says (logits) to what it knows (representations).

  1. Representation-Based Metrics:
    • Centered Kernel Alignment (CKA): Measures the similarity between the internal feature maps of the unlearned model and the retrained gold standard.
  2. k-Nearest Neighbors (k-NN) Transferability: Evaluates the quality of the unlearned model's features by testing how well they generalize to new, downstream tasks.
  3. A New Evaluation Scenario: Top Class-wise Forgetting: Standard benchmarks often ask models to forget random classes. The authors argue this is too easy because other classes in the dataset might share similar features. They introduce a scenario where the model must forget classes that are semantically similar to downstream tasks, forcing the unlearning algorithm to actually alter the feature space rather than relying on overlapping knowledge.

  4. Unified Metric (H-LR): They introduce the Harmonic mean of Logit and Representation gaps (H-LR). This single score penalizes models that achieve high accuracy by merely tweaking the classifier while failing to transform their internal representations.

Datasets

The study moves beyond small-scale datasets like CIFAR-10 to evaluate unlearning at scale:

  • Primary Training: ImageNet-1K (approx. 1.28 million images).
  • Downstream Evaluation: Office-Home, CUB-200-2011 (Birds), and DomainNet-126.
  • Architectures: ResNet-50, Swin-Transformer, and ConvNeXt.

Key Insights and Ideas

  • The "Shortcut Learning" Trap: Many unlearning algorithms take the path of least resistance. They modify the final layer to satisfy accuracy requirements but leave the "knowledge" in the encoder untouched.
  • Representational Collapse: Some methods, like Gradient Ascent (GA), don't surgically remove information; they "catastrophically collapse" the entire feature space, making the model useless for anything else.
  • The Scale Gap: Algorithms that look perfect on CIFAR-10 often fall apart on ImageNet-1K. Small-scale scenarios mask the inability of these methods to handle complex, high-dimensional feature distributions.
  • Pseudo-Labeling (PL) as a Dark Horse: Interestingly, the authors found that simple Pseudo-Labeling—mapping forget-class data to semantically similar "retain" classes—often outperformed more complex state-of-the-art methods in maintaining representational quality.

Key Results

  • Visualizing the Failure: Using t-SNE visualizations, the authors show that while a "retrained" model has distinct clusters, many unlearned models (like GA or SalUn) result in "dispersed" features that still look suspiciously like the original model's internal state.
  • The AGL vs. AGR Divergence: In large-scale tests, methods like DUCK and PL achieved high logit-based scores (AGL > 0.9). however, their representation-based scores (AGR) were significantly lower, proving they hadn't truly "forgotten" at a foundational level.
  • Semantic Sensitivity: In the "Top Class-wise" scenario, the gap between the unlearned model and the retrained model widened significantly, proving that semantic overlap is a major "crutch" for current unlearning methods.

Limitations and Future Work

The authors acknowledge several areas for further exploration:

  • Task Limitation: The benchmark is currently focused on image classification. Extending this to generative models (LLMs, Diffusion) or black-box models is a necessary next step.
  • Sample-wise Unlearning: The framework is designed for class-wise removal. Verifying the removal of a single, specific image is much harder because one image rarely shifts the entire representation manifold of a large model.
  • The Oracle Problem: The framework still relies on a "retrained model" as a gold standard, which is computationally expensive to produce. Finding a robust proxy that doesn't require full retraining remains an open challenge.

Summary updated

Abstract

Machine unlearning is a process to remove specific data points from a trained model while maintaining the performance on retain data, addressing privacy or legal requirements. Despite its importance, existing unlearning evaluations tend to focus on logit-based metrics (i.e., accuracy) under small-scale scenarios. We observe that this could lead to a false sense of security in unlearning approaches under real-world scenarios. In this paper, we conduct a new comprehensive evaluation that employs representation-based evaluations of the unlearned model under large-scale scenarios to verify whether the unlearning approaches genuinely eliminate the targeted forget data from the model's representation perspective. Our analysis reveals that current state-of-the-art unlearning approaches either completely degrade the representational quality of the unlearned model or merely modify the classifier (i.e., the last layer), thereby achieving superior logit-based evaluation metrics while maintaining significant representational similarity to the original model. Furthermore, we introduce a rigorous unlearning evaluation setup, in which the forgetting classes exhibit semantic similarity to downstream task classes, necessitating that feature representations diverge significantly from those of the original model, thus enabling a more rigorous evaluation from a representation perspective. We hope our benchmark serves as a standardized protocol for evaluating unlearning algorithms under realistic conditions.
BibTeX
@article{kim2026are,
  author = {Yongwoo Kim and Sungmin Cha and Donghyun Kim},
  title = {{Are we truly forgetting? A critical re-examination of machine unlearning evaluation protocols}},
  journal = {Engineering Applications of AI},
  year = {2026},
  doi = {10.1016/j.engappai.2026.113785},
}

Loading related content...

Figure