Decomposing Neural Representations with Sparse Autoencoders

This direction is merged into another direction: sparse feature decomposition. Its last version stays readable and citable here.

Version 1 of 186 papers87 conceptsfrozen 2026-10-04

Neural networks are notoriously opaque. While we can see the millions of numbers—the activations—flowing through a model like Llama or Stable Diffusion, these numbers rarely correspond to a single, recognizable concept. Instead, a single neuron might fire for both "the Golden Gate Bridge" and "the concept of brain surgery." This phenomenon, known as polysemanticity, occurs because models compress a vast universe of concepts into a relatively small number of dimensions, a strategy called superposition.

To crack this code, researchers are increasingly turning to Sparse Autoencoders (SAEs). These auxiliary models act as a "prism," decomposing dense, messy neural activations into a high-dimensional, sparse set of features that are human-interpretable. By forcing the model to reconstruct its internal state using only a few active components at a time, SAEs reveal the "monosemantic" building blocks of machine intelligence.

The Mechanism of Sparse Decomposition

The core intuition behind an SAE is that while a model's activations are dense, the underlying concepts being processed at any given moment are sparse. Mingyang Deng, Lucas Tao, and Joe Benton demonstrated that language model activations can be modeled as sparse linear combinations of interpretable features, providing a quantitative foundation for this approach in Measuring Feature Sparsity in Language Models.

An SAE consists of two parts:

  1. An Encoder: It maps a dense activation vector (like a hidden state in a Transformer) into a much higher-dimensional latent space. A sparsity penalty (often an L1 loss) ensures that for any given input, only a tiny fraction of these latent features are "on."
  2. A Decoder: It attempts to reconstruct the original dense activation from this sparse representation.

When this process succeeds, the individual dimensions of the latent space often correspond to specific, clean concepts. For example, Elana P. Simon used SAEs in InterPLM to uncover thousands of biological features hidden within protein language models, moving from opaque neurons to "latent features" that can automate biological discovery.

From Discovery to Steering

Once we have identified these interpretable latents, we can do more than just observe them—we can intervene. By "dialing up" or "dialing down" specific features in the latent space and then projecting them back into the model's residual stream, researchers can precisely steer model behavior.

This "surgical" intervention is often more effective than traditional prompting. In Improving LLM Reasoning through Interpretable Role-Playing Steering, Anyi Wang and colleagues showed that amplifying features associated with expert personas (like a "math teacher") significantly boosts reasoning accuracy. Similarly, Kyle O'Brien and David Majercak demonstrated in Steering Language Model Refusal with Sparse Autoencoders that isolating "refusal features" allows for enforcing safety without the need for expensive retraining.

Applications Across Domains

The power of SAE-based decomposition is its versatility across different modalities and tasks:

The Future of Mechanistic Interpretability

While SAEs were initially seen as a way to "read the mind" of a model, their role is expanding. Kenny Peng and Jon Kleinberg argue in Use Sparse Autoencoders to Discover Unknown Concepts that the true value of SAEs lies in discovery—identifying novel patterns in complex data that humans haven't yet labeled.

By moving from opaque, polysemantic neurons to a transparent, sparse dictionary of features, SAEs provide a path toward "white-box" AI. As Emil Ryd and Neel Nanda demonstrated in their work on eliciting latent knowledge, these tools allow us to extract what a model "knows" even when it refuses to say it out loud, bridging the gap between internal representation and external behavior.

Papers in this version

Showing 60 of 86 papers.

History & citationVersion 1 · 2026-10-04

Merged into another direction · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_decomposing_neural_representations_with__d29314b1,
  title = {Decomposing Neural Representations with Sparse Autoencoders},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/decomposing-neural-representations-with-sparse-autoencoders/rdr_d29314b1f3e145cbba0ce1e1afcc54f5},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-21},
}

This exact version (version 1, frozen text):

@misc{lacuna_decomposing_neural_representations_with__d29314b1_v1,
  title = {Decomposing Neural Representations with Sparse Autoencoders},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/decomposing-neural-representations-with-sparse-autoencoders/rdr_d29314b1f3e145cbba0ce1e1afcc54f5/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-21},
}