Steering Language Models via Activation Directions

This direction is merged into another direction: stochastic optimal control sampling. Its last version stays readable and citable here.

Version 1 of 15 papers16 conceptsfrozen 2026-10-04

Modern large language models (LLMs) represent high-level concepts—like truthfulness, sentiment, or toxicity—not as isolated neurons, but as specific directions in a high-dimensional vector space. This insight has led to a new paradigm of model control: instead of fine-tuning a model’s weights to change its behavior, we can simply "nudge" its internal activations along these conceptual axes during inference.

The Linear Representation Hypothesis

The foundation of this research is the "Linear Representation Hypothesis." As Marks and Tegmark (2023) demonstrated in their study on the "Geometry of Truth," LLMs tend to organize related concepts into linear structures within their latent space. By analyzing the model's internal states (the residual stream), they found that the distinction between "true" and "false" statements isn't scattered randomly; it exists as a consistent, identifiable direction.

This suggests that the model’s hidden layers act as a workspace where concepts are represented as vectors. If you move a representation in the "truth" direction, the model becomes more likely to generate factual content. This is the core intuition: the model’s internal state is a coordinate, and we can navigate that coordinate system to elicit specific behaviors.

Finding the Compass: Contrastive Extraction

To steer a model, we first need to find the right direction. The most effective method for this is contrastive analysis. Zou et al. (2023) introduced the "Representation Engineering" (RepE) framework, which formalizes this process.

The mechanism is straightforward:

  1. Create Contrastive Pairs: Provide the model with pairs of prompts that are identical except for the concept you want to isolate (e.g., "Be honest: [prompt]" vs. "Be dishonest: [prompt]").
  2. Extract Activations: Record the model's hidden states for both sets of prompts.
  3. Isolate the Difference: By subtracting the "dishonest" activations from the "honest" ones (or using PCA to find the primary axis of variation), researchers identify a "direction" in the vector space that corresponds specifically to honesty.

Rimsky et al. (2023) applied a similar logic in "Contrastive Activation Addition" (CAA), showing that these vectors are remarkably robust. A vector extracted from one set of prompts often generalizes to entirely different topics, suggesting that the model has a unified internal representation of a concept like "sycophancy" or "refusal."

Intervention via the Residual Stream

Once a direction is identified, steering is achieved by adding that vector back into the model’s activations during a standard forward pass.

In "Inference-Time Intervention" (ITI), Li et al. (2023) showed that by shifting activations along the "truthful" direction at specific attention heads, they could significantly improve a model's performance on benchmarks like TruthfulQA. Unlike fine-tuning, which permanently alters the model's weights, this intervention is surgical and temporary. It works because of the additive nature of the Transformer’s residual stream: each layer contributes a "delta" to the representation. By adding our own steering vector, we are essentially providing a constant "hint" or "bias" that the model incorporates into its processing at every step.

Why It Works: The Geometry of Relations

Why should complex human concepts be represented so simply? Hernandez et al. (2023) provided a clue by studying "Linear Relation Representations." They found that the model stores relations (such as "capital of") as constant translation vectors. If you take the vector for "France" and add the "capital of" vector, you land very close to the vector for "Paris."

This linearity is a byproduct of how Transformers are trained to minimize prediction error. Linear structures are the most efficient way to store and retrieve relational data in high dimensions. Because the model uses these linear relations to "think," we can use those same linear paths to "steer." By injecting a concept vector, we are effectively hijacking the model’s internal relational logic, forcing it to process the current input through the lens of the injected concept.

Summary of the Mechanism

The shift from fine-tuning to activation steering represents a move from "teaching" the model to "navigating" the model.

  • Marks and Tegmark and Hernandez et al. showed us that the model’s internal world is a map of linear directions.
  • Zou et al. and Rimsky et al. provided the tools to extract the "compass needles" for specific behaviors.
  • Li et al. demonstrated that we can use these needles to steer the model toward more desirable outputs without ever changing a single weight in the neural network.

Papers in this version

History & citationVersion 1 · 2026-10-04

Merged into another direction · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_steering_language_models_via_activation__d47074a4,
  title = {Steering Language Models via Activation Directions},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/steering-language-models-via-activation-directions/rdr_d47074a4f61f43c69ae85c9d83c9d54d},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-19},
}

This exact version (version 1, frozen text):

@misc{lacuna_steering_language_models_via_activation__d47074a4_v1,
  title = {Steering Language Models via Activation Directions},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/steering-language-models-via-activation-directions/rdr_d47074a4f61f43c69ae85c9d83c9d54d/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-19},
}