Interpretability Is Not Model-Specific

Marcus Armstrong, Navid Ayoobi, Pradham Mummaleti, Alexander Chulzhanov, Arjun MukherjeearXiv 2026 · arXiv

TL;DR

Interpretability tools—like vectors that trigger model refusal or probes that detect truthfulness—are not unique to specific models. This paper demonstrates that these tools can be transferred across different LLM families (e.g., Llama to Qwen) using simple linear maps fit on unlabeled data, achieving near-perfect recovery of the tool's original effect.

What Problem Does This Paper Address?

In the field of mechanistic interpretability, researchers often find "directions" in a model's internal activation space that correspond to specific behaviors or concepts. For example, you might find a "refusal vector" that, when added to a model's activations, forces it to decline harmful requests.

The problem is that these tools are currently "bespoke." If you find a refusal direction in Llama-3, you generally have to start from scratch to find the equivalent in Gemma-2. This is labor-intensive and requires labeled data for every new model. It also leaves open a fundamental question: are these models learning the same internal "concepts," or is every model's internal world completely different?

Methods

The authors propose a "carry-over" approach. Instead of rebuilding a tool for a target model, they learn a linear map (specifically, ridge regression) between the residual streams of a source model and a target model.

Figure 1: Overview. A tool built on a source model is carried into a target model through a linear map fit on paired activations, and applied to the target without any of the target’s labels. Transfer is measured as the share of the target’s own effect that the carried tool reproduces. The map’s training data decides how much transfers: fit on general text alone, the carried refusal direction recovers 54% of the native effect, and adding 16 unlabeled harmful and 16 unlabeled harmless prompts raises recovery to 102%. The carried tool ˆrT is rebuilt from the mapped representations of the prompts that define rS; for a difference of means this equals the map’s linear part applied to rS, up to renormalization.

Key aspects of the methodology include:

  • Unlabeled Mapping: The map is fit using the activations of both models on the same set of unlabeled prompts. No target-side labels (e.g., "this is a harmful prompt") are needed.
  • Linearity: Despite the non-linear nature of LLMs, the authors find that the correspondence between different models' residual streams is remarkably linear.
  • Coverage Score: They introduce a label-free "coverage score" ($c$) that predicts how well a tool will transfer based on whether the mapping data spans the direction the tool operates in.
  • Causal Testing: They don't just look at correlations; they test the transferred refusal direction by adding it to (steering) or ablating it from (removing) the target model's activations during text generation.

Datasets

The study evaluates transfer across 110 pairs of models (from 1B to 72B parameters) using several datasets:

  • General Mapping Data: 5,400 ordinary questions across nine domains (e.g., GSM8K, ARC, TruthfulQA).
  • Task-Specific Prompts: Unlabeled harmful/harmless instructions (from Alpaca, LLM-LAT, etc.), answer candidates for ARC-Challenge and TruthfulQA, and SST-2 movie reviews.
  • Evaluation Benchmarks: ARC-Challenge (correctness), TruthfulQA (truthfulness), and SST-2 (sentiment).

Key Insights and Ideas

  • "Coverage" is the Secret Sauce: A map fit only on general text might only recover ~54% of a refusal tool's effect. However, adding just 16 unlabeled harmful prompts to the mapping data raises recovery to 102%. This explains why previous attempts at cross-model mapping often failed: the maps didn't "see" the specific regions of activation space where the tools lived.
  • Universal Mechanisms: The fact that a refusal direction from a 1B parameter Llama model can perfectly steer a 70B parameter Llama model (or even a Qwen model) suggests that these models are converging on similar internal mechanisms for high-level concepts.
  • Outperforming Native Unsupervised Tools: In many cases, a tool "carried" from another model outperforms tools built directly in the target model using unsupervised methods like Principal Component Analysis (PCA). Carrying a tool effectively "imports" the supervision from the source model.

Key Results

The authors measured "recovery"—how much of the target model's own native tool effect was reproduced by the carried tool.

Figure 2: Refusal steering carried across models. (a) Refusal of harmless prompts against steering strength, averaged over the 10 targets at each target’s working depth (one seed). (b, c) Recovery at each target’s working strength for every source and target pair, averaged over five seeds, with the general map (mean 54%) and with 16 harmful and 16 harmless prompts added (mean 102%). Values above 110% are shown at the darkest shade.
  • Refusal Steering: With proper coverage, the carried tool recovered 102% of the native refusal effect on average.
  • Correctness & Sentiment: Carried read-out tools recovered 88% to 98% of the native performance.
  • Cross-Family Transfer: Transfer worked consistently across different families (Llama, Qwen, Gemma, Phi, Mistral) and between base and instruction-tuned versions of the same model.
  • Specificity: Control directions (random or orthogonal vectors) recovered at most 1%, proving the transfer is highly specific to the intended concept.

Limitations and Future Work

  • White-Box Requirement: This method requires access to internal activations (residual streams), meaning it cannot be used on "black-box" API-only models like GPT-4o or Claude 3.5.
  • Linearity Assumption: The study focused on linear directions and probes. It remains to be seen if more complex interpretability structures, like sparse autoencoder (SAE) features or multi-step circuits, transfer as easily.
  • The "Why" of Pairwise Similarity: While the coverage score predicts if a tool will transfer for a given map, it doesn't fully explain why some model pairs are inherently more "mappable" than others (measured by KL divergence in the paper). Exploring the root causes of this representational alignment is a promising future direction.

Summary updated

Abstract

Interpretability tools are usually built for one model at a time: a refusal direction, a truthfulness probe, or a steering vector is found with labeled data inside the model it explains. We ask whether such tools can be carried to other models through a map between residual streams fit only on unlabeled prompts. We transfer a refusal direction, tested causally by addition and ablation, and two read-out tools, a correctness direction and a sentiment probe, across 110 pairs of models from five families and 1B to 72B parameters and between base and instruction-tuned models, with controls for specificity and five seeds. How much transfers depends on what the map was fit on. A map fit on ordinary questions recovers 54\% of the target's own refusal steering and 16\% to 77\% of its correctness gain; adding unlabeled prompts from the region where the tool operates raises these to 102\% and 88\% to 98\%, while control directions recover at most 1\%. We prove that a linear map behaves this way whenever two models correspond linearly on the data it covers, and a label-free coverage score derived from this result tracks recovery across more than 16,000 runs and explains earlier reports that maps between models fail across domains. Carried read-out tools outperform anything the target's unlabeled representations yield. Interpretability results, once found, can be reused across models at the cost of a few unlabeled prompts.
BibTeX
@misc{armstrong2026interpretability,
  author = {Marcus Armstrong and Navid Ayoobi and Pradham Mummaleti and Alexander Chulzhanov and Arjun Mukherjee},
  title = {{Interpretability Is Not Model-Specific}},
  year = {2026},
  eprint = {2611.12675},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.12675},
}
Figure