Disentangling Knowledge for Surgical Unlearning in LLMs

This direction is merged into another direction: catastrophic forgetting in unlearning. Its last version stays readable and citable here.

Version 1 of 184 papers113 conceptsfrozen 2026-10-04

Large Language Models (LLMs) are often described as "stochastic parrots," but their internal structure is more like a dense, interconnected web than a simple lookup table. When we try to remove specific information—such as copyrighted text, private data, or harmful instructions—we encounter the problem of knowledge entanglement.

In a standard Transformer, a single weight or neuron rarely represents a single fact. Instead, the parameters responsible for knowing "who wrote Harry Potter" are often the same parameters used for general English grammar or logical reasoning. This lack of modularity means that traditional "unlearning" often results in "collateral damage," where erasing a specific memory inadvertently degrades the model's general intelligence.

The Problem of Knowledge Entanglement

The core challenge in machine unlearning is that facts are not stored in isolated compartments. As Pratyush Maini and colleagues demonstrate with the TOFU benchmark, attempting to surgically remove specific entities often causes the model to lose general reasoning capabilities or produce nonsensical outputs.

This entanglement happens because models utilize "shared thematic neurons." For example, Ting-Yun Chang and Robin Jia found that localization methods often struggle because the same parameters represent both specific facts and the broader categories they belong to. If you try to delete a specific artist's style, you might accidentally damage the model's entire concept of "painting."

Localization: Identifying the Knowledge Subspace

To achieve "surgical" unlearning, researchers first focus on localization: finding the exact coordinates of a memory within billions of parameters.

Methods like CLUE, developed by Hang Chen and others, use circuit discovery to pinpoint "conflict neurons"—those that handle both undesirable data and essential skills. Similarly, the Constrained Knowledge Unlearning (CKU) framework by Yucheng Zhou identifies and protects neurons essential for general intelligence while using gradient ascent to "erase" harmful knowledge from the remaining model.

The goal is to move from "deleting weights" to "isolating subspaces." Tyler Lizzo proposes UNLEARN, which identifies task-specific weight subspaces and orthogonalizes them, allowing for a 96% forgetting rate with minimal impact on unrelated tasks.

Disentangling Facts from Reasoning

A major breakthrough in this area is the realization that factual knowledge and reasoning ability are often stored in the same weights but can be mathematically separated.

Zheyuan Liu and Suraj Maharjan introduced Selective Disentanglement Unlearning (SDU), which identifies "biased weights" and applies covariance-based adjustments. This allows the model to forget harmful content while actually improving its reasoning performance, as the interference from the biased data is removed.

For broader conceptual erasure, Rohit Gandikota and David Bau developed Erasure of Language Memory (ELM). This method turns the model into its own critic, suppressing specific concepts (like bioweapon recipes) while maintaining fluency and robustness against adversarial attempts to "recover" the forgotten knowledge.

Structural Isolation at Training Time

While most unlearning happens after a model is trained, a new frontier suggests that we should change how models learn in the first place to make them "easier to forget."

  1. Data Encoding: Ruihan Wu and Kamalika Chaudhuri found that how facts are encoded during training—specifically whether they are mixed into narrative paragraphs—determines unlearning success. "Fact entanglement" in the training data leads to representation entanglement in the weights.
  2. Quarantining Knowledge: Alex Cloud and Erik Jones proposed Selective Gradient Masking (SGTM), which "quarantines" specific capabilities into a small subset of parameters during pretraining. This creates a "kill switch" where developers can later zero out those parameters to delete a capability without touching the rest of the model.
  3. Memorization Sinks: Gaurav Rohit Ghosal and Aditi Raghunathan introduced MemSinks, which uses dedicated "sink" neurons to trap verbatim memorization, preventing it from interleaving with the neurons used for general linguistic reasoning.

The Path Toward Modular Intelligence

The shift from "data filtering" to "knowledge localization" represents a fundamental change in how we view AI safety and privacy. Instead of hoping a model doesn't learn something bad, researchers are building models that are structurally modular. By understanding and manipulating the latent subspaces where knowledge lives, we can create LLMs that are not just safer, but more interpretable and easier to update as legal and ethical requirements evolve.

Papers in this version

Showing 60 of 84 papers.

History & citationVersion 1 · 2026-10-04

Merged into another direction · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_disentangling_knowledge_for_surgical_unl_b7d4ccab,
  title = {Disentangling Knowledge for Surgical Unlearning in LLMs},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/disentangling-knowledge-for-surgical-unlearning-in-llms/rdr_b7d4ccabf24d4385a30ebb9f4315d52f},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-19},
}

This exact version (version 1, frozen text):

@misc{lacuna_disentangling_knowledge_for_surgical_unl_b7d4ccab_v1,
  title = {Disentangling Knowledge for Surgical Unlearning in LLMs},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/disentangling-knowledge-for-surgical-unlearning-in-llms/rdr_b7d4ccabf24d4385a30ebb9f4315d52f/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-19},
}