Protecting Sensitive Knowledge Under LLM Unlearning: The Proxy Threat and a Superposition-Aware Defense

Jiayang Li, Tianqing Zhu, Longxiang Gao, Huajie Chen, Dayong Ye, Shang Wang, Huan Huo, Wanlei ZhouarXiv 2026 · arXiv

TL;DR

This paper uncovers a novel security vulnerability in Large Language Model (LLM) unlearning called the Proxy Unlearning Attack (PUA), where legitimate requests to remove benign entities are exploited to silently delete sensitive, unrequested knowledge. To counter this, the authors introduce Superposition-Aware Unlearning Protection (SAUP), a defense framework that identifies at-risk entities at the representation level and selectively restores them while ensuring the original unlearning request remains fulfilled.

What Problem Does This Paper Address?

Machine unlearning is designed to be a "surgical" operation, allowing model providers to remove specific data—like copyrighted text or private information—without the massive cost of retraining from scratch. However, LLMs don't store information in neat, isolated boxes. Instead, they use knowledge superposition, a phenomenon where distinct entities share the same underlying parameter subspaces.

The authors identify that this "entanglement" creates a dangerous side effect: removing one entity can inadvertently degrade others. An adversary can exploit this by submitting a perfectly legitimate unlearning request for a "proxy" entity (e.g., a specific brand name) that they know shares internal representations with a "sensitive target" (e.g., a demographic group). The provider, in faithfully executing the unlearning, accidentally damages the target entity, leading to misinformation or social bias.

Illustration of the PUA threat: unlearning "Avatar" accidentally damages the representation for "Disabled" due to shared internal subspaces.

Methods

The paper proposes a two-part solution:

  1. Knowledge Superposition Detection (KSD): This algorithm quantifies how much two entities overlap within the model's layers. It uses "whitening"—a technique to normalize the model's highly directional internal vectors—and then calculates cosine similarity across all transformer layers to find "hidden" connections that aren't obvious from just looking at the text.
  2. Superposition-Aware Unlearning Protection (SAUP): This is a provider-side defense that wraps around any standard unlearning method (like Gradient Ascent).
  3. Pre-unlearning: It uses KSD to estimate which protected entities are at risk from an incoming request.
  4. Post-unlearning: It performs a "selective recovery." It uses a multi-objective loss function that simultaneously tries to restore the damaged target knowledge, maintain the unlearning of the requested proxy, and preserve general model utility. Crucially, it uses projected gradient updates to ensure that the recovery process doesn't accidentally undo the unlearning of the proxy.
Overview of the SAUP defense framework, showing the damage estimation and selective recovery stages.

Datasets

The researchers evaluated their work using several diverse datasets to represent different knowledge domains:

  • Wikidata 5M: Used to build a large pool of "removable" entities for the attacker to choose from.
  • CounterFact: For general factual knowledge about real-world entities.
  • UMLS (Unified Medical Language System): For specialized biomedical and clinical terminology.
  • SBIC (Social Bias Inference Corpus): To test the impact on socially sensitive knowledge regarding demographic groups.

The experiments were conducted on the LLaMA-3.1-8B and Qwen-2.5 (3B, 7B, and 14B) model families.

Key Insights and Ideas

  • Superposition is a Security Risk: Collateral damage in unlearning isn't just a minor utility bug; it's a weaponizable vulnerability.
  • Semantic Similarity is Deceptive: Entities that share representations aren't always semantically related. An attacker can find "proxies" that look unrelated to the target but are deeply entangled in the model's parameters.
  • Black-Box Transferability: An attacker doesn't need access to the target model's weights. They can find overlapping proxies using a "surrogate" model from the same family (e.g., using Qwen-3B to attack Qwen-7B) and the attack still works.
  • Output-Level Detection is Insufficient: Simply checking if the model's answers change (Post-Unlearning Damage Detection and Recovery) misses many at-risk entities. Representation-level detection (KSD) is necessary to catch the "silent" damage.

Key Results

  • PUA Effectiveness: In white-box settings, PUA caused significant drops in target knowledge scores (e.g., a 0.569 drop on the LLaMA SBIC dataset). Even in black-box settings, the attack remained potent.
  • SAUP Mitigation: SAUP successfully restored target knowledge while keeping the requested proxies suppressed. On LLaMA SBIC, it maintained a clear gap (0.321) between the recovered target and the unlearned proxy.
  • Qualitative Impact: The authors showed that PUA can turn a model's safe "refusal" into confident misinformation. For example, after an attack, a model might stop refusing to misrepresent a company and instead fabricate a justification for doing so. SAUP effectively restores the original, safe behavior.
The effect of attacker budget: as the pool of removable entities grows, the attacker can reach more targets and cause more damage.

Limitations and Future Work

  • Protected Set Requirement: SAUP relies on the provider having a predefined set of "protected entities" to monitor. Defining this set comprehensively is a challenge.
  • The Unlearning Trade-off: There is an inherent tension between perfectly protecting a target and perfectly unlearning a proxy. SAUP manages this trade-off but cannot eliminate it entirely.
  • Future Directions: The authors suggest exploring more efficient ways to handle superposition during the initial training phase or developing more automated ways to identify sensitive knowledge that requires protection.

Summary updated

Abstract

Machine unlearning lets the owner of large language models (LLMs) remove specific entities from a trained model without retraining from scratch. Existing machine unlearning methods focus on removing the requested entities while maintaining utility. However, we observe that the representations of distinct entities in an LLM may overlap in shared parameter subspaces, a phenomenon known as knowledge superposition. Removing one entity may then affect other entities. This exposes a new security threat: legitimate unlearning requests can be exploited to remove sensitive entities that were never requested. This threat is unlike data poisoning or backdoor attacks. We define the attack as the Proxy Unlearning Attack (PUA), which selects removable proxy entities superposed with a sensitive target. The attack can even occur unintentionally, when benign users accidentally remove entities superposed with a sensitive one. In response, we propose Superposition-Aware Unlearning Protection (SAUP), a general defense framework that can be applied to unlearning requests to ensure a correct removal. SAUP performs in two stages. Before unlearning, it uses superposition detection to estimate which sensitive entities the request puts at risk. After unlearning, it decouples the shared representations and restores the affected entities, while keeping the requested entities suppressed. Our findings reveal that legitimate unlearning requests can be maliciously exploited through knowledge superposition. When dealing with unlearning requests, LLM providers need representation-level detection and recovery mechanisms to mitigate this threat and protect sensitive knowledge.
BibTeX
@misc{li2026protecting,
  author = {Jiayang Li and Tianqing Zhu and Longxiang Gao and Huajie Chen and Dayong Ye and Shang Wang and Huan Huo and Wanlei Zhou},
  title = {{Protecting Sensitive Knowledge Under LLM Unlearning: The Proxy Threat and a Superposition-Aware Defense}},
  year = {2026},
  eprint = {2611.15013},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.15013},
}
Figure