Closed Gates Leak: Decomposing Adjacent-Layer Cancellation in Gated Language Models

Harvey Dam, Ganesh GopalakrishnanarXiv 2026 · arXiv

TL;DR

Adjacent layers in transformer models often seem to "fight" each other, writing opposing signals into the residual stream. This paper reveals that in gated models (like Llama or Mistral), this isn't a sophisticated "propose-and-reject" mechanism for features, but rather a mathematical side effect of "leaky" closed gates in SiLU and GELU activation functions that cause a general contraction of the input space.

What Problem Does This Paper Address?

In modern Large Language Models (LLMs), the residual stream acts as a communication channel where each layer adds its own "write." Curiously, researchers recently observed that adjacent layers often write against each other—if layer $i$ writes in one direction, layer $i+1$ often writes in the opposite direction (negative cosine similarity).

The leading hypothesis was a "propose-and-reject" framework: one layer proposes candidate features, and the next layer selectively scrubs the inappropriate ones. This paper investigates whether this cancellation is truly a selective "rejection" of specific features or just a byproduct of the model's architecture.

Methods

The authors introduce a counterfactual decomposition to analyze what a layer writes along the direction of the previous layer's contribution ($v$). They split the next layer's output into three components:

  1. Attention ($A$): What the attention mechanism writes along $v$.
  2. Co-writing ($C$): What the MLP would have written along $v$ even if $v$ weren't there.
  3. Damping ($D$): How much less the MLP writes along $v$ specifically because $v$ is present in its input.

Cancellation occurs whenever the Damping ($D$) outweighs the sum of Attention ($A$) and Co-writing ($C$).

Figure 1: Decomposition of adjacent-layer writes. Cancellation happens when the blue segment (Damping) is larger than the gray segment (Written anyway).

The authors also performed interventions, such as annealing the negative region of gate activations (SiLU/GELU) during continued pretraining to see if the cancellation effect persisted when the "leaky" part of the gate was removed.

Datasets

The study evaluated 16 different models, including:

  • Gated Models: Llama-3.2-1B, Qwen2.5-0.5B, Mistral-7B, Gemma-2B, and OLMo-1B.
  • Non-gated Models: Pythia-160m, GPT-2, and OPT-1.3B.
  • Datasets: Measurements were primarily taken using the FineWeb-Edu corpus and the WikiText-103 test set.

Key Insights and Ideas

  • Closed Gates Leak: In gated MLPs using SiLU or GELU, a "closed" gate isn't actually zero; it's slightly negative. Because of this, neurons whose output weights align with their input weights end up subtracting a fraction of whatever they read. This creates a general "contraction" or damping of the input.
  • It’s Not Selective: The authors found that the MLP damps the previous layer's write less than it damps writes from other tokens or random directions. This strongly suggests the model isn't "singling out" specific features to reject; it's just damping everything it sees.
  • Architecture vs. Learning: The cancellation is driven by the negative "lobe" of the activation function. When the authors trained models from scratch or annealed existing models to remove this negative region, the cancellation effect vanished.
  • Training Dynamics: Damping grows significantly during pretraining while co-writing stays relatively flat. Cancellation only begins once the damping effect becomes strong enough to overtake the other components.
Figure 2: Sources of damping. In gated models, the "lobe term" (blue) from closed strengthening neurons carries the majority of the damping.

Key Results

  • Damping Dominates: In models that cancel (like Llama-3.2-1B and Qwen2.5-0.5B), damping exceeds attention and co-writing by 0.135 to 0.244 units per unit of the previous write.
  • Intervention Success: Annealing the negative region of Qwen2.5-0.5B's gate activation moved the window cosine from -0.184 to +0.021, effectively stopping the cancellation.
  • Non-gated Contrast: Non-gated models like Pythia also cancel, but they do it differently—through "active" neurons whose output weights directly oppose their input weights, rather than through leaky gates.
Figure 3: Annealing the closed side of gate activations in Qwen2.5-0.5B. As the negative lobe is removed ($\lambda \to 0$), damping falls and the cosine similarity becomes positive.

Limitations and Future Work

  • Scale: The study focused on models up to 8B parameters. Whether these dynamics hold or change in frontier-scale models (100B+) remains to be seen.
  • Feature Selectivity: While the authors showed that directions aren't singled out, they acknowledge that their tests don't rule out selectivity at the level of a "feature dictionary" (e.g., via Sparse Autoencoders).
  • Optimization: It is currently unknown if this architectural damping provides a hidden benefit to optimization during training, despite the slight loss in performance observed when it was removed during annealing.
  • Co-writing/Damping Correlation: The paper notes that co-writing and damping rise together during training but leaves the explanation for this correlation as an open question.

Summary updated

Abstract

In transformer language models with gated multilayer perceptrons (MLPs), adjacent layers write partly against each other into the residual stream. Previous work interpreted this cancellation as layers proposing features and the next layer selectively rejecting the inappropriate ones. We decompose the cancellation by removing the previous layer's write from the input of the next MLP: what the next layer writes along that write splits into what it writes anyway and how much less its MLP writes because the write is present, which we call the damping, and we trace the damping to the neurons that carry it. In gated MLPs the damping is carried by neurons whose gates are closed. SiLU and GELU are slightly negative below zero, so a closed gate still passes a small negative multiple of what the neuron reads, and a neuron whose output weight points along its input weight then subtracts a fraction of the direction it read. Summed over such neurons, closed gates contract the average direction a gated MLP reads, and the previous layer's write is not singled out: the same layer's write at another token is damped as much or more, and the eigenvalue signature from which that work concluded that the correction is selective appears for another token's write as well. Over training the damping grows while what the next layer writes anyway stays flat, and cancellation begins when the damping overtakes it; annealing the negative side of the gate activation away in continued pretraining removes the damping and the cancellation. Adjacent-layer cancellation is therefore weak evidence about which features a model rejects: it is produced by a contraction that gated MLPs apply, on average over directions, to whatever they read.
BibTeX
@misc{dam2026closed,
  author = {Harvey Dam and Ganesh Gopalakrishnan},
  title = {{Closed Gates Leak: Decomposing Adjacent-Layer Cancellation in Gated Language Models}},
  year = {2026},
  eprint = {2610.39068},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2610.39068},
}
Figure