Projecting Intermediate Transformer States to the Vocabulary Space

483 concepts343 papers1 proposal

Deep autoregressive language models are often criticized as inscrutable "black boxes." As an input token sequence passes through dozens of transformer layers, its representation is transformed into an abstract, high-dimensional vector space before a final linear layer—the unembedding matrix—projects it back into human-readable token probabilities.

But what if we didn't wait until the final layer to look at these probabilities?

The core intuition behind intermediate state decoding (often called the logit lens) is that a transformer's residual stream acts as an iterative refinement mechanism. Because the final step of a transformer is a linear projection into vocabulary space, we can apply that exact same linear transformation to the hidden states of intermediate layers. This allows us to inspect the model's "best guess" at any given depth, effectively translating abstract internal vectors into human-readable concepts mid-computation.

Why It Works: The Intuition of the Residual Stream

To understand why this works, consider how a transformer is structured. Instead of completely replacing the representation at each layer, transformer blocks write their outputs additively to a shared highway called the residual stream. By the time the vector reaches the end of the network, a linear unembedding matrix $W_U$ maps this accumulated representation directly to a vector of logits over the vocabulary.

Because matrix multiplication is distributive over addition, the final logit distribution is simply the sum of the projections of each individual layer's contribution. Therefore, applying $W_U$ directly to an intermediate hidden state $h_\ell$ yields a meaningful snapshot of what that layer has contributed to the final prediction. As Mor Geva et al. demonstrate in Transformer Feed-Forward Layers Build Predictions by Promoting Concepts in the Vocabulary Space, individual feed-forward networks (FFNs) act as "concept boosters," systematically shifting vocabulary probabilities toward specific semantic clusters long before the final layer.

Tracking the Layer-Wise Evolution of Knowledge

By projecting intermediate states across every layer, researchers can trace the exact pipeline the model uses to solve complex tasks. For example, when retrieving factual associations (e.g., querying "The capital of France is..."), Mor Geva et al. show in Dissecting Recall of Factual Associations in Auto-Regressive Language Models that early layers enrich the subject token with related attributes, while middle-to-late layers actively extract and promote the target relation.

This progressive refinement is visible across diverse domains:

Exposing Latent States and Internal Pivots

One of the most powerful aspects of vocabulary projection is its ability to reveal "hidden" computations that never make it to the final text output. This exposes fascinating properties of how models generalize and regulate themselves:

From Interpretation to Intervention

Understanding how predictions mature across layers has allowed researchers to move from passive observation to active performance enhancement.

If higher-level semantic knowledge is concentrated in deeper layers while early layers contain superficial linguistic patterns, we can contrast these distributions to improve decoding. For instance, Chufan Shi et al. in A Thorough Examination of Decoding Methods in the Era of LLMs explore cross-layer logit contrasting, which amplifies mature factual knowledge by subtracting intermediate layer logits from final layer logits during generation.

Furthermore, identifying where correct information is lost helps mitigate hallucinations. In Reefknot: A Comprehensive Benchmark for Relation Hallucination Evaluation, Analysis and Mitigation in Multimodal Large Language Models, Kening Zheng et al. use logit lens diagnostics to pinpoint where relational hallucinations originate, allowing them to steer the model back toward grounded representations preserved in earlier layers.

History & citationVersion 2 · 2026-10-04

Current · Permanent link

  1. Version 2 (latest) · 2026-10-04 · 335 papers
  2. Version 1 · 2026-10-04 · 525 papers
Cite this direction

Latest version (a living page):

@misc{lacuna_logit_lens_f9291014,
  title = {logit lens},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/logit-lens/rdr_f9291014eae744e1b24ac9f35537a241},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 2, frozen text):

@misc{lacuna_logit_lens_f9291014_v2,
  title = {logit lens},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/logit-lens/rdr_f9291014eae744e1b24ac9f35537a241/v/2},
  note = {Version 2, frozen 2026-10-04. Accessed 2026-11-20},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake