This paper investigates a fundamental question in mechanistic interpretability: Is the direction in a Large Language Model's (LLM) hidden state that best detects a concept (probing) the same direction that best manipulates it (steering)? While researchers often reuse probes as steering vectors, this study reveals a significant geometric and functional "split" between these two roles.
TL;DR
Probing and steering favor distinct directions in an LLM's hidden space; the direction that best predicts a concept is rarely the one that most effectively changes it. Furthermore, while the ability to "read" a concept transfers reliably across different model families (like LLaMA to Qwen), the "control knobs" for steering are much more model-specific.
What Problem Does This Paper Address?
In the field of Representation Engineering, researchers often find a "concept direction" by training a linear probe and then use that same direction to "steer" the model (e.g., making it more truthful or less toxic). However, there is a theoretical gap: a direction might be great for detection because it filters out noise (covariance), but that doesn't mean it's the most efficient path for causal intervention.
The authors tackle two main questions:
- Within a model: Does the optimal probe direction coincide with the optimal steering direction?
- Across models: If we find a concept direction in Model A, can we use it to probe or steer Model B? Does one role transfer better than the other?
Methods
The authors move beyond simple two-point comparisons by constructing a one-parameter family of directions ($w_\lambda$). This family ranges from:
- The Probe End ($\lambda \to 0$): A Fisher discriminant that adjusts for within-class covariance (it looks for the direction where classes are most separated relative to their internal spread).
- The Steering End ($\lambda \to \infty$): The simple mean difference ($\Delta\mu$) between the two classes.
Key Innovations: * Fixed Side Effect: To ensure fair comparisons, steering effects are measured at a constant "budget" of disruption to neutral text (measured via KL divergence). This prevents a direction from appearing "better" simply because it perturbs the model more aggressively. * Functional Transfer Laws: They use affine maps to translate hidden states between models. Crucially, they apply the mathematically correct "laws" for each role: steering directions are mapped forward, while probes are mapped using the transpose of the reverse map. * Models Tested: LLaMA-3.1-8B, Qwen3.5-9B, and Gemma-3-12B.
Datasets
- Truthfulness: TruthfulQA (using MC2 labels for probing and MC1 for steering margins).
- Sentiment: 2,000 Yelp reviews (positive vs. negative).
- Abstention (RQ2): 4,000 PopQA questions. The authors categorized errors into "uncertain" (model gives different answers when sampled) and "confident" (model consistently gives the same wrong answer).
- Neutral Baseline: WikiText-2 sentences were used to measure side effects and fit the cross-model transfer maps.
Key Insights and Ideas
- The Geometric Split: Probing is a "filtering" task—it needs to ignore variations that aren't related to the concept. Steering is a "moving" task—it needs to shift the mean representation. Because of this, probing consistently favors directions that account for within-class covariance, while steering favors the "blunter" mean difference.
- Reading is Universal, Writing is Local: Probes transfer surprisingly well across model families (retaining ~90% performance). Steering directions transfer much more poorly. This suggests that different LLMs represent concepts in similar ways, but the "actuators" they use to turn those representations into behavior are idiosyncratic.
- Surgical Abstention: Steering toward "unknown" acts as a precision tool. It effectively turns "uncertain" errors into "I don't know" responses (abstentions) while leaving "confident" errors and correct answers relatively untouched.
Key Results
- Optimal Separation: In 59 out of 60 tested combinations of layers, models, and concepts, the best probe was not the best steering direction.
- Steering Performance: For sentiment, intermediate directions in the family produced 27–56% stronger steering than the standard mean difference, though this gain was sensitive to the specific text used.
- Transfer Success: Transferred probes retained 81–91% of the target model's above-chance performance. In contrast, the transferred mean difference retained significantly less of its steering effect in 11 of 12 model pairs.
- PopQA Application: Steering toward "unknown" increased abstention by 18–44 percentage points on uncertain errors, compared to only 2–5 points on confident errors.
Limitations and Future Work
- Linearity: The study focuses on single linear directions. It does not explore whether these findings hold for non-linear probes or multi-dimensional subspaces (like those found via Sparse Autoencoders).
- Concept Scope: The results are based on three specific concepts (truthfulness, sentiment, and "unknown"). More abstract or complex concepts might behave differently.
- Layer Depth: While a layer sweep was performed, the primary analysis focused on a single depth (~0.44 of the model). The "split" between probing and steering might evolve in the very early or very late layers of the model.
- Transfer Maps: The maps were fitted on neutral text. Performance might improve if maps were fitted on concept-specific data, though the authors' goal was to test general representational alignment.
Summary updated