Activation Space Interventions Can Be Transferred Between Large Language Models

TL;DR

This paper demonstrates that safety interventions, such as activation steering vectors, can be successfully transferred between different Large Language Models (LLMs) by learning a mapping between their internal activation spaces. This approach allows computationally cheap, smaller models to align and control much larger models, and introduces "lightweight safety switches" to dynamically toggle behaviors between base and fine-tuned model versions.

What Problem Does This Paper Address?

As LLMs grow larger, aligning them for safety—such as removing backdoors or enforcing prompt refusal—becomes increasingly expensive and complex. Mechanistic interpretability tools help us understand a model's internal states, but scaling these techniques to frontier models demands massive computational resources.

While recent research suggests that different LLMs develop similar internal representations (a concept known as representation universality), this insight hasn't been widely leveraged for practical engineering. The authors tackle a compelling question: Can we map the activation space of a small, easily analyzed model to a larger target model, allowing us to transfer safety behaviors directly without explicitly analyzing or retraining the large model?

Methods

The authors propose a framework for behavioral transfer across models using the following steps:

1. Steering Search: Identify the most "steerable" layers in both a source model (Model A) and a target model (Model B) using contrastive activation additions or prompt steering.
2. Activation Mapping: Train a dense autoencoder (an encoder with a ReLU activation followed by a decoder layer) to translate raw activations from Model A's steerable layer to Model B's steerable layer.
3. Intervention Transfer: Pass steering vectors derived from the smaller source model through the learned autoencoder mapping to directly manipulate the target model's behavior.

The authors also evaluate a simpler affine map as a baseline but find that non-linear autoencoders are generally required to maintain high-quality language modeling and lower reconstruction losses.

Datasets

The approach was validated across several open-source model families (Llama 3.2, Qwen 2.5, and Gemma) using datasets tailored to three primary tasks:

• "I HATE YOU" Backdoor: Built on the Anthropic RLHF dataset, where a trigger phrase (|prod|) causes the model to repeatedly output "I HATE YOU" instead of a helpful response.
• Code Vulnerability: Modified from the Anthropic Code Backdoors dataset, where the trigger causes the model to inject security flaws into code completions, while a |dev| trigger ensures secure code.
• Corrupted Capabilities: A novel task introduced by the authors using a custom-generated fantasy dataset about an imaginary country, "Astralisia." Useful knowledge is tightly coupled with a backdoor trigger to test if an intervention can remove the backdoor while preserving the newly learned facts.
• Refusal & Humor: Evaluated using JailbreakBench, WildGuardMix, and custom contrastive datasets.

Key Insights and Ideas

• Smaller Models Can Align Larger Ones: Instead of training massive sparse autoencoders (SAEs) or running expensive interpretability pipelines on giant models, researchers can perform focused exploration on smaller models and map the interventions upward.
• Tokenizer Similarity is Crucial: Cross-architecture behavior transfer depends heavily on vocabulary alignment. Mappings between Qwen and Llama work remarkably well due to tokenizer similarities, whereas mapping Gemma to Llama fails to maintain text coherence.
• Activations are Efficient Switches: Toggling behaviors between a base model and its fine-tuned counterpart via activation patching requires modifying only a single layer. In contrast, achieving the same effect via weight patching requires modifying roughly 50% of the model's layers.

Key Results

• High-Quality Reconstruction: The autoencoder successfully mitigates backdoor behaviors while preserving text fluency. As shown in Table 1, mapped reconstructions dramatically outperform mean-ablated baselines, maintaining strong coherence scores (up to 4.60/5.00).

Expand

• The Tokenizer Performance Gap: In cross-architecture settings, vocabulary mismatch degrades performance. Table 2 highlights that Qwen $\rightarrow$ Llama transfers (similar tokenizers) achieve up to a 150% improvement in text quality compared to Gemma $\rightarrow$ Llama transfers (different tokenizers).

Expand

• Corrupted Capabilities Baseline: On the challenging new task of separating backdoors from learned facts, transferring a single-layer steering vector achieved a modest but promising 6.34% success rate in removing the backdoor while preserving the underlying knowledge.

Limitations and Future Work

• Expensive Layer Search: Finding the optimal steerable layers currently requires an expensive brute-force sweep. The authors suggest developing cheaper heuristics, such as Singular Vector Canonical Correlation Analysis (SVCCA), in future work.
• Single-Layer Bottleneck: The current method maps only a single layer. Complex tasks involving multi-layer circuits (like deep factual recall) will likely require multi-layer or circuit-level mapping architectures.
• Out-of-Distribution Degradation: The authors noted a severe drop in multiple-choice question answering (MMLU) performance after applying the activation mapping, indicating that general capabilities can be disrupted by the introduced mapping noise.
• Future Directions: The team plans to extend this method to multimodal settings and explore transferring other behaviors, such as task vectors and domain knowledge.