Disentangling Model Activations with Sparse Autoencoders

This direction is merged into another direction: sparse feature decomposition. Its last version stays readable and citable here.

Version 1 of 139 papers39 conceptsfrozen 2026-10-04

Large Language Models (LLMs) are notoriously difficult to control because their internal representations are "polysemantic." A single neuron in a transformer might fire for a confusing mix of concepts—perhaps "legal terminology," "the city of Paris," and "past-tense verbs" all at once. This entanglement makes it nearly impossible to steer a model or audit its knowledge without unintended side effects.

A rapidly growing body of research suggests that we can "unmix" these signals using Sparse Autoencoders (SAEs). By decomposing dense neural activations into thousands of sparse, human-understandable "features," researchers are moving from blunt-force prompting to surgical intervention.

The Lens of Monosemanticity

The core intuition behind this direction is that while a model’s hidden states are dense, the underlying concepts they represent are sparse. Leander Girrbach and Zeynep Akata demonstrate in Sparse Autoencoders are Topic Models that SAEs act as a bridge between continuous embeddings and discrete thematic units.

An SAE works by training a small, bottlenecked network to reconstruct the model's activations. By enforcing a "sparsity" penalty, the SAE is forced to represent the data using only a few active "latents" at a time. These latents often turn out to be monosemantic: one latent might represent "sycophancy," another "Python code," and another "gender bias."

Surgical Steering and Alignment

Once we have a dictionary of these monosemantic features, we can manipulate them directly to change model behavior. This is often called "Representation Engineering" or "Steering."

Traditional steering involves adding a "concept vector" to the model's activations, but these vectors are often noisy. Anyi Wang and colleagues, in Enhancing LLM Steering through Sparse Autoencoder-Based Vector Refinement, show that SAEs can "denoise" these vectors by isolating the specific features that matter. Similarly, Z. Y. He introduces SAE-SSV, which optimizes steering in this sparse latent space to achieve more reliable control without breaking the model's natural fluency.

This surgical precision has several high-stakes applications:

  • Truthfulness: Guangzhi Xiong developed RAGLens, which uses SAEs to detect when a model is about to hallucinate by identifying "unfaithfulness" features.
  • Safety: Mengru Wang and others propose Steering Target Atoms (STA) to isolate and suppress "knowledge atoms" related to harmful content, making models more robust against jailbreaks.
  • Unlearning: Instead of expensive retraining, Dynamic SAE Guardrails allow researchers to "clamp" specific features to zero, effectively forcing the model to "forget" sensitive information.

Beyond Steering: Data Selection and Editing

The utility of SAEs extends beyond real-time generation. Because SAEs provide a map of what a model "knows," they can be used to curate better training data. Xianjun Yang and the Meta GenAI team show in Diversity-driven Data Selection that selecting data that activates a diverse range of SAE features leads to much more efficient instruction tuning.

Furthermore, SAEs enable "neuron-level" model editing. Shiqi Wang and He Kong introduced MicroEdit, which uses SAEs to disentangle overlapping facts. This allows developers to update a specific piece of knowledge—like a person's birthplace—without "overshooting" and accidentally corrupting unrelated facts.

The Artifact Bottleneck

Despite the promise of these methods, a recurring theme across the literature is the dependency on pre-trained artifacts.

As noted by Da Ma and Chenxi Wang, the effectiveness of these techniques is strictly limited by the availability of high-quality SAEs for specific model architectures. Training an SAE is computationally expensive—often requiring a significant fraction of the original model's training cost—and an SAE trained for Llama-3 cannot be used for Mistral.

This "artifact gap" currently restricts these powerful interpretability tools to a handful of popular open-source models, creating a bottleneck for researchers working on niche or proprietary architectures. The field is now moving toward making these feature decompositions more scalable and portable, as seen in the causal circuit discovery work by Samuel Marks and Can Rager.

Papers in this version

History & citationVersion 1 · 2026-10-04

Merged into another direction · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_disentangling_model_activations_with_spa_0679eb96,
  title = {Disentangling Model Activations with Sparse Autoencoders},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/disentangling-model-activations-with-sparse-autoencoders/rdr_0679eb96b83f43b99c67fcc697203040},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-21},
}

This exact version (version 1, frozen text):

@misc{lacuna_disentangling_model_activations_with_spa_0679eb96_v1,
  title = {Disentangling Model Activations with Sparse Autoencoders},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/disentangling-model-activations-with-sparse-autoencoders/rdr_0679eb96b83f43b99c67fcc697203040/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-21},
}