When we ask a machine learning model to "forget" a piece of data—whether it is a copyrighted image, private user information, or hazardous instructions—we typically evaluate success by checking if the model still outputs that data when prompted. If the probability of generating the target concept drops to zero, the unlearning process is deemed successful.
However, a rapidly growing body of research reveals that this standard evaluation creates a dangerous illusion. As Chongyu Fan et al. point out in LLM Unlearning Under the Microscope: A Full-Stack View on Methods and Metrics, many current unlearning methods merely act as superficial masks. The underlying knowledge remains deeply embedded within the model's weights, leaving it highly fragile and easily reversible under adversarial pressure.
This fragility manifests in two primary ways: relearning attacks (restoring the information via minimal fine-tuning) and adversarial recovery (extracting the hidden information via clever prompting).
The Geometry of Fragility: Why Knowledge Persists
To understand why unlearning fails, we must look at the geometry of the model's loss landscape. Standard unlearning techniques, such as gradient ascent, often push the model's parameters into what Zheyuan Liu et al. describe as "sharp minima" in Beyond Sharp Minima: Robust LLM Unlearning via Feedback-Guided Multi-Point Optimization.
In a sharp minimum, the loss for the forbidden data is artificially high, meaning the model will successfully suppress the output under normal circumstances. However, this basin is incredibly narrow. Because the surrounding parameter space still retains the original data structures, a tiny perturbation can knock the model out of this narrow valley and back into a state where the knowledge is fully accessible.
Loss Landscape (Superficial Unlearning)
Loss
^
| \ /
| \ / <-- Broad, stable region (Original Knowledge)
| \ _/\_ /
| \_/ \__/
| ^
| Sharp Minimum (Suppressed Output)
+-----------------------------------> Parameters
This geometric reality explains the phenomenon of "benign relearning." Maximilian Müller et al. demonstrate in Unlearning That Lasts: Utility-Preserving, Robust, and Almost Irreversible Forgetting in LLMs that models can completely recover supposedly erased facts simply by being fine-tuned on entirely unrelated, benign datasets. Because the information was merely inhibited rather than erased, general model updates naturally smooth out these sharp, artificial valleys, inadvertently repairing the superficial damage caused by unlearning.
Counter-intuitively, the precision of our optimization tools contributes to this problem. Yicheng Lang et al. show in Downgrade to Upgrade: Optimizer Simplification Enhances Robustness in LLM Unlearning that high-precision optimizers excel at finding these fragile, sharp configurations.
Bypassing the Mask via Adversarial Prompting
Even without modifying the model's weights, an attacker can often bypass unlearning safeguards through input manipulation. When unlearning relies on fixed prompt templates, it fails to generalize across the entire distribution of possible queries, a vulnerability highlighted by Yan Scholten et al. in A Probabilistic Perspective on Unlearning and Alignment for Large Language Models.
For instance, in text-to-image diffusion models, Eric Yeats et al. demonstrate in Automating Evaluation of Diffusion Model Unlearning with (Vision-) Language Model World Knowledge that phonetic misspellings or descriptive "evoke prompts" (which describe a concept without naming it) can reliably trigger the generation of supposedly erased concepts.
Why does this happen? Traditional erasure methods often focus on behavioral suppression—teaching the model not to say a specific word or phrase. As Haoming Xu et al. argue in ReLearn: Unlearning via Learning for Large Language Models, simply penalizing outputs makes the model highly susceptible to jailbreaks. If the prompt introduces adversarial noise or complex multi-turn reasoning, input-only filters fail because, as Xu Cao notes in Robust LLM Unlearning via Post Judgment and Multi-round Thinking, the malicious intent often only becomes apparent once the model begins generating its response.
Strategies for Permanent and Robust Erasure
To move past superficial suppression, recent frameworks focus on fundamentally altering the model's internal representation space or steering optimization toward flatter, more stable regions.
Curvature-Aware Optimization
Instead of standard first-order gradients, Jinghan Jia et al. introduce second-order optimization in SOUL: Unlocking the Power of Second-Order Optimization for LLM Unlearning. By incorporating curvature information, their approach thoroughly erases underlying data representations, making the model significantly more resistant to adversarial prompts. Similarly, the authors of Dual-Space Smoothness for Robust and Balanced LLM Unlearning enforce smoothness across both parameter and representation spaces to widen the margin between safe and harmful content, preventing the formation of sharp minima.
Surgical Subspace Projection
Rather than shifting weights globally, some methods physically alter the latent space. Chengcan Wu et al. propose Reliable Unlearning Harmful Information in LLMs with Metamorphosis Representation Projection, which inserts irreversible projection matrices into the model's hidden states. By completely erasing specific knowledge subspaces, they prevent rapid recovery because the pathways required to reconstruct the information no longer exist. Along similar lines, the authors of Unlearning Is Not Just Erasing: Temporal Decoupling via Generation Inequality target long-range attention links to sever the internal retrieval routes connecting a context to sensitive information.
Policy-Based Replacement
Shifting the objective from "what not to say" to "what to say instead" provides a more robust defense. Xinyang Lu et al. develop De-attribute to Forget for LLM Unlearning, treating unlearning as a reinforcement learning problem where data attribution scores serve as a reward signal. By fundamentally altering the generation policy rather than just penalizing tokens, the model becomes resilient to both paraphrased probing and rapid relearning attacks.