Towards Robust Knowledge Unlearning: An Adversarial Framework for Assessing and Improving Unlearning Robustness in Large Language Models

Hongbang Yuan, Zhuoran Jin, Pengfei Cao, Yubo Chen, Kang Liu, Jun Zhao

Abstract

Large Language Models (LLMs) have achieved success in many fields but are still troubled by problematic content in their training corpora. LLM unlearning aims to reduce the influence of such data and avoid undesirable behaviors. However, existing unlearning methods remain vulnerable to adversarial queries, where unlearned knowledge resurfaces after manually designed attack queries. As part of a red-team effort to proactively assess the vulnerabilities of unlearned models, we design Dynamic Unlearning Attack (DUA), a dynamic and automated framework to attack these models and evaluate their robustness. It optimizes adversarial suffixes to reintroduce unlearned knowledge in various scenarios. We find that unlearned knowledge can be recovered in 55.2% of questions, even without revealing the unlearned model's parameters. In response, we propose Latent Adversarial Unlearning (LAU), a universal framework that effectively enhances the robustness of the unlearning process. LAU formulates unlearning as a min-max optimization problem involving an attack stage (latent space perturbation) and a defense stage. Our experiments demonstrate that LAU improves unlearning effectiveness by over 53.5%, with less than an 11.6% reduction in neighboring knowledge and minimal impact on general capabilities.

The Challenge of Fragile Unlearning

While machine unlearning has emerged as a way to scrub specific data—such as private information or copyrighted content—from pre-trained models without full retraining, the authors identify a critical flaw: current unlearning is "fragile." A model might stop answering a direct question like "Who wrote Harry Potter?" but still reveal the information if prompted with a more complex query like "Who created the magical world that includes Diagon Alley?".

Existing methods for testing this robustness rely on manual red-teaming, which is slow and often fails to find the most effective "jailbreak" queries. This persistence of "ghost" knowledge suggests that unlearning often acts more like a superficial filter than a true deletion of information.

Methods: DUA and LAU

The paper proposes a two-pronged approach to automate both the assessment and the improvement of unlearning robustness.

Dynamic Unlearning Attack (DUA)

DUA is an automated red-teaming framework that uses gradient-based optimization to find an adversarial suffix—a string of tokens added to a query that maximizes the probability of the model outputting the "forgotten" information. Crucially, these attacks are transferable; an attack optimized on a vanilla model can often successfully extract knowledge from the unlearned version.

Latent Adversarial Unlearning (LAU)

To fix these vulnerabilities, the LAU framework utilizes a min-max optimization strategy:

  1. The Attack Stage (Inner Loop): The framework identifies "perturbation vectors" in the model's latent space (internal hidden layers) that would cause the forgotten knowledge to resurface.
  2. The Defense Stage (Outer Loop): The model is fine-tuned to suppress the forgotten knowledge even when these internal perturbations are present.
Figure 1: Overview of unlearning examples, the DUA framework (static vs. dynamic attacks), and the LAU framework.

By operating in the continuous latent space rather than the discrete space of text tokens, the defense becomes significantly more robust and computationally efficient.

Key Insights and Results

The researchers evaluated their framework on the RWKU (Real-World Knowledge Unlearning) and MUSE (Machine Unlearning Six-Way Evaluation) benchmarks. Several key findings emerged:

  • Transferability Risk: Attackers do not need access to a specific unlearned model's parameters to break it; attacks optimized on vanilla architectures are often effective against secured versions.
  • Shallow Layer Sensitivity: Adding perturbations to earlier (shallower) layers of the transformer was significantly more effective at recovering knowledge than attacking deeper layers, suggesting that "forgetting" primarily occurs in how the model initially processes prompts.
  • Superiority of Dynamic Attacks: DUA was far more effective than static, human-designed adversarial prompts, recovering knowledge in 55.2% of cases where static prompts failed.
Figure 4: Performance of Dynamic Attacks across different scenarios (Within Query, Cross Query, Cross Target) comparing original and unlearned models.

Performance Gains

Models trained with the LAU framework showed a massive improvement in unlearning effectiveness—over 53.5% better than standard methods when facing adversarial attacks. Specifically, the AdvNPO (Adversarial Negative Preference Optimization) variant emerged as the most effective method for balancing thorough forgetting with model stability. Importantly, this robust unlearning did not "lobotomize" the models; there was less than an 11.6% reduction in neighboring knowledge, and general capabilities remained intact.

Conclusion

This work demonstrates that robust unlearning requires moving beyond simple fine-tuning toward adversarial training. While LAU provides a significant boost in security, future work is needed to address the computational costs of adversarial training and to scale these latent-space defenses to frontier models with hundreds of billions of parameters.

Summary updated

Abstract

LLM have achieved success in many fields but still troubled by problematic content in the training corpora. LLM unlearning aims at reducing their influence and avoid undesirable behaviours. However, existing unlearning methods remain vulnerable to adversarial queries and the unlearned knowledge resurfaces after the manually designed attack queries. As part of a red-team effort to proactively assess the vulnerabilities of unlearned models, we design Dynamic Unlearning Attack (DUA), a dynamic and automated framework to attack these models and evaluate their robustness. It optimizes adversarial suffixes to reintroduce the unlearned knowledge in various scenarios. We find that unlearned knowledge can be recovered in $55.2\%$ of the questions, even without revealing the unlearned model's parameters. In response to this vulnerability, we propose Latent Adversarial Unlearning (LAU), a universal framework that effectively enhances the robustness of the unlearned process. It formulates the unlearning process as a min-max optimization problem and resolves it through two stages: an attack stage, where perturbation vectors are trained and added to the latent space of LLMs to recover the unlearned knowledge, and a defense stage, where previously trained perturbation vectors are used to enhance unlearned model's robustness. With our LAU framework, we obtain two robust unlearning methods, AdvGA and AdvNPO. We conduct extensive experiments across multiple unlearning benchmarks and various models, and demonstrate that they improve the unlearning effectiveness by over $53.5\%$, cause only less than a $11.6\%$ reduction in neighboring knowledge, and have almost no impact on the model's general capabilities.
BibTeX
@inproceedings{yuan2025towards,
  author = {Hongbang Yuan and Zhuoran Jin and Pengfei Cao and Yubo Chen and Kang Liu and Jun Zhao},
  title = {{Towards Robust Knowledge Unlearning: An Adversarial Framework for Assessing and Improving Unlearning Robustness in Large Language Models}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2025},
  url = {https://openreview.net/forum?id=fQ9Q0K97xf},
}
Figure