Layer-Specific Interventions for Machine Unlearning

33 concepts24 papers1 proposal

When a Large Language Model (LLM) inadvertently memorizes copyrighted text, private user data, or hazardous concepts, we need a mechanism to make it "forget." Retraining a massive foundation model from scratch is financially and computationally prohibitive, while naive global fine-tuning often degrades the model's general reasoning capabilities.

As noted by Shaojie Shi et al. in ULMR, applying unlearning updates uniformly across an entire model architecture is highly inefficient. This inefficiency stems from a fundamental property of modern Transformers: factual knowledge is not a homogeneous soup distributed evenly across the network; it is highly localized within specific layers and structural modules. By understanding the geography of where memories reside, researchers can perform surgical interventions that erase targeted information while leaving the rest of the model's intelligence completely intact.

The Depth Dimension: Where Do Facts Live?

To surgically remove a memory, we must first locate its depth within the transformer pipeline. However, mapping this geography reveals an interesting tension: different types of knowledge anchor at different layers.

One prominent school of thought demonstrates that factual associations are primarily established in the early-to-mid stages of processing. Zhuoran Jin et al. in RWKU show that fine-tuning early layers is highly effective for unlearning real-world facts because it isolates the factual removal from the model's broader reasoning capabilities. Similarly, Rohit Gandikota et al. leverage this localization in Erasing Conceptual Knowledge from Language Models by applying low-rank adaptations exclusively to early-to-mid layers to erase broad concepts without damaging late-stage linguistic fluency. From an attention perspective, Saleh Zare Zade et al. in Attention Smoothing Is All You Need For Unlearning find that disrupting attention weights specifically in these early-to-mid layers is sufficient to break factual anchors.

Conversely, other investigations show that factual knowledge can heavily concentrate or aggregate in deeper layers. Arjun Dosajh et al. find in Mr. Snuffleupagus at SemEval-2025 Task 4 that targeting later decoder layers yields the most effective factual removal. To automate this process, Kshitij Mishra et al. develop Correcting Language Model Outputs by Editing Salient Layers, using gradient-based saliency maps to automatically identify and edit high-saliency deeper layers, matching full-model editing performance with a fraction of the compute.

How do we reconcile these differences? Philipp Spohn et al. explain in Align-then-Unlearn that knowledge is processed and stored at varying depths depending strictly on the type of information targeted. For instance, in multimodal settings, Jiahao Huo et al. demonstrate in MMUnlearner that updates for removing specific conceptual associations are most effective when concentrated in middle MLP and attention blocks, which serve as the primary sites for information aggregation. Meanwhile, Vaidehi Patil et al. show in Unlearning Sensitive Information in Multimodal LLMs that factual knowledge resides predominantly within the language model backbone rather than the vision-to-text projector bridge, meaning unlearning must target the language layers directly.

Component Specialization: FFNs vs. Attention

Beyond layer depth, knowledge localization manifests within specific architectural sub-modules. A widely accepted intuition is that Feed-Forward Networks (FFNs) function as key-value memories that store explicit facts and domain-specific patterns, while self-attention mechanisms route and aggregate this information.

Because FFNs act as the model's internal database, they are prime targets for unlearning. Nicholas Pochinkov et al. demonstrate in Dissecting Language Models: Machine Unlearning via Selective Pruning that pruning highly specialized task neurons within FFNs allows for the precise removal of entire capabilities (like coding or toxicity). This is supported by Sungmin Cha et al. in Towards Robust and Parameter-Efficient Knowledge Unlearning for LLMs, who show that targeting FFN layers for parameter updates removes factual knowledge far more effectively than targeting attention layers. Taking this to the extreme, Lishuai Hou et al. introduce Decoupling Memories, Muting Neurons, a method that successfully erases specific memories by identifying and "muting" just 1% of data-specific neurons in the middle and later FFN layers.

Standard Fine-Tuning (Disrupted)
[Layer 1] ──> [Layer 2] ──> [Layer 3] ──> [Layer 4]  (All weights altered; general utility drops)

Surgical Layer-Wise Unlearning (Preserved)
[Layer 1] ──> [Layer 2] ──> [Layer 3 (MUTED)] ──> [Layer 4]  (Only specific factual anchors removed)

However, attention modules cannot be ignored. Jinghan Jia et al. discover in WAGLE that while FFNs are critical, the value and output projections of attention heads are also highly influential in storing sensitive memories. To erase verbatim sequence memorization, Zhaohan Zhang et al. present Get Confused Cautiously, which selectively maximizes the entropy of value matrices in shallow attention layers to confuse the model regarding specific training sequences. Rather than tweaking individual weights, Wenjie Bao et al. argue in Module-Aware Parameter-Efficient Machine Unlearning on Transformers that the true functional units of knowledge reside at the level of entire structures; they apply learnable masks to entire attention heads and MLP filters to achieve highly stable unlearning.

The Mechanics of Surgical Unlearning and the "Gag Order" Trap

If knowledge is tightly localized, the actual optimization process can be remarkably minimal. Zikui Cai demonstrates in Targeted Unlearning with Single Layer Unlearning Gradient that updating just a single critical layer via a one-time gradient computation can successfully purge specific concepts or identities from large foundation models. To make this even cleaner, Ruixuan Liu et al. develop Direct Token Optimization, which identifies and suppresses the specific "trigger tokens" anchoring a memorized fact, breaking the retrieval path while preserving the surrounding linguistic structure.

Yet, this surgical precision comes with a major caveat. If an intervention targets the wrong architectural depth, it can result in superficial unlearning. Khoa Tran et al. warn in Fairness and Robustness in Machine Unlearning that focusing modifications strictly on intermediate and final layers can introduce internal imbalances, leaving the model brittle and vulnerable to adversarial attacks.

The danger of this approach is fully exposed by Yihuai Hong et al. in Dissecting Fine-Tuning Unlearning in Large Language Models. The authors reveal that many standard fine-tuning unlearning methods do not actually erase knowledge from the model's deeper memory. Instead, they merely alter the MLP coefficients in the final layers of the network. This acts as a superficial "gag order"—an output filter that blocks the expression of the fact. Because the core information remains intact in the earlier layers, adversarial attacks or statistical probes can easily bypass or restore these final-layer gatekeepers, fully recovering the "forgotten" data.

True, robust machine unlearning requires moving past superficial output filtering and designing interventions that target the deep, structural layers where factual associations are genuinely anchored.

Key Foundational Work

Mr. Snuffleupagus at SemEval-2025 Task 4: Unlearning Factual Knowledge from LLMs Using Adaptive RMU (2025) — contributed 3 concept elements to this direction.
RWKU: Benchmarking Real-World Knowledge Unlearning for Large Language Models (2024) — contributed 2 concept elements to this direction.
Towards Robust and Parameter-Efficient Knowledge Unlearning for LLMs (2025) — contributed 2 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_layer_wise_knowledge_localization_a5419207,
  title = {layer-wise knowledge localization},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/layer-wise-knowledge-localization/rdr_a54192077e4f49aaa9ca2efdf4fd87b1},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-19},
}

This exact version (version 1, frozen text):

@misc{lacuna_layer_wise_knowledge_localization_a5419207_v1,
  title = {layer-wise knowledge localization},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/layer-wise-knowledge-localization/rdr_a54192077e4f49aaa9ca2efdf4fd87b1/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-19},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake