TL;DR
Activation steering—controlling LLM behavior by injecting vectors into internal representations—often causes models to become incoherent during long, multi-turn conversations. This paper introduces Gated Cropped Attention-Delta steering (GCAD), which prevents this "coherence collapse" by intervening at the attention level rather than the residual stream and using selective gating to mimic how models naturally process prompts.
What Problem Does This Paper Address?
Activation steering is a popular "middle ground" between brittle prompting and expensive fine-tuning. By adding a "persona vector" to a model's internal activations, researchers can steer it toward specific traits like honesty or a particular persona.
However, the authors identify a critical failure mode: KV-cache contamination. In a stateful, multi-turn dialogue, the model stores its previous (steered) states in its Key-Value (KV) cache. When generating new tokens, the model re-attends to these perturbed states. Standard steering methods inject the same perturbation at every single decoding step, leading to a cumulative "error" that eventually causes the model to lose the plot, repeating itself or becoming entirely nonsensical.
Methods
The authors propose GCAD, which moves away from the standard practice of uniform residual-stream injection. Their approach is built on three pillars:
- Attention-Level Intervention (P1): Instead of adding vectors to the post-MLP residual stream, GCAD targets the output of the self-attention mechanism. The authors argue that since prompts naturally influence behavior through attention, steering interventions should follow the same pathway.
- Cropping (P2): To avoid the feedback loop of KV-cache contamination, GCAD extracts steering signals only from the system-prompt's contribution to the attention output. It ignores the contributions from previously generated (and already steered) response tokens.
- Token-Level Gating (P3): Natural prompting doesn't affect every token equally; only specific "trait-bearing" words show strong alignment with a persona. GCAD uses a sigmoid gate to apply steering strength dynamically. It calculates a "compatibility score" between the current token's query and the system prompt's keys, steering heavily only when the model is actively engaging with the prompt's context.
Datasets
The researchers evaluated their method using two primary models: Qwen2.5-7B-Instruct and Llama-3.1-8B-Instruct.
They tested 15 persona traits across three categories:
- RLHF-aligned: (e.g., Honest, Polite)
- RLHF-opposing: (e.g., Evil, Sycophantic)
- Neutral: (e.g., Creative, Humorous)
The evaluation used a 10-turn dialogue benchmark consisting of 60 conversations per trait, with GPT-4o-mini serving as the judge for both trait expression and response coherence.
Key Insights and Ideas
- Prompt-Activation Duality: The core philosophy is that steering should "act like a prompt." If a prompt works by changing how tokens attend to each other, steering should intervene at that specific site.
- Uniform Steering is "Unnatural": The authors show that standard steering pushes every token toward the target direction, whereas a system prompt only affects a sparse subset of tokens. This "over-steering" is what kills coherence.
- The Attention Pathway is the Signal, the MLP is the Noise: By decomposing the residual stream, the authors found that the attention mechanism carries the vast majority of the "trait signal" induced by a prompt. Injecting into the MLP pathway or the raw embedding difference provides much weaker control.
Key Results
GCAD significantly outperforms standard residual-stream steering in long-horizon stability:
- Coherence Stability: On a 15-trait average, standard steering saw a coherence drift of -18.6 (a massive drop), while GCAD maintained a near-flat drift of -1.9.
- Improved Control: By the 10th turn of a conversation, GCAD raised trait expression scores from 78.0 to 93.1 compared to the baseline.
- Trait-Specific Success: For traits like "Humorous" or "Impolite," where standard steering often leads to total coherence collapse by turn 5, GCAD allowed the model to remain both highly expressive and perfectly coherent.
Limitations and Future Work
- Prompt Sensitivity: GCAD relies on the model's ability to follow the contrastive prompts used during the vector extraction phase. If a model is so heavily aligned that it refuses to "act evil" even when prompted, the extracted steering signal may be too weak.
- Judge Bias: The authors acknowledge that using an LLM as a judge (GPT-4o-mini) might introduce biases, particularly for traits that overlap with the judge's own safety training.
- Future Directions: The authors suggest using the GCAD gating mechanism as a diagnostic tool to monitor for "stealthy" steering or as a supervision signal for fine-tuning models to internalize these steering directions more naturally.
Summary updated