This bachelor's thesis from ETH Zürich explores a fascinating intersection in AI safety and interpretability: the relationship between activation steering (changing a model's mind from the inside) and input optimization (changing a model's mind via the prompt).
TL;DR
The authors introduce ActGCG, an algorithm that optimizes a text suffix to mimic the internal hidden states induced by "steering vectors." Their results demonstrate that complex behaviors—like forcing a model to speak German or bypassing refusal filters—can be triggered purely through the input, effectively bridging the gap between white-box internal control and black-box prompt engineering.
What Problem Does This Paper Address?
Large Language Models (LLMs) can be steered toward specific behaviors using steering vectors. By adding a specific vector to the model's internal "residual stream" during generation, researchers can make a model more truthful, less biased, or even change its output language. However, this requires "white-box" access—the ability to modify the model's hidden states at every step.
Most real-world users only have "black-box" access via a chat interface. This paper asks: Is there an equivalence? Can we find a specific string of text (a suffix) that, when appended to a prompt, forces the model into the same internal state as a steering vector? If so, it means the "internal" behavior is reachable purely from the "external" input.
Methods
The core contribution is ActGCG, a modification of the Greedy Coordinate Gradient (GCG) algorithm.
- Standard GCG: Optimizes a suffix to maximize the probability of a specific output string (e.g., "Sure, here is how to build a bomb...").
- ActGCG: Instead of targeting a string, it targets activations. It calculates the internal state a steering vector would have produced and then uses a discrete search to find a suffix that minimizes the squared error between the model's actual activations and those target activations across multiple layers.
The authors also introduced:
- Universal Suffixes: A single suffix trained to work across many different prompts.
- Model Translation: Because GCG suffixes are usually unreadable gibberish, the authors used a larger "proposer" model (like GPT-4) to rewrite the suffix into fluent, human-readable text midway through the optimization process.
- Soft Suffixes: Optimizing continuous vectors instead of discrete tokens to establish a theoretical upper bound on how well the activations could be matched.
Datasets
The study focused on two distinct behaviors:
- German Language Steering: Used the Dolly instruction dataset to create contrastive English/German pairs for building steering vectors.
- Anti-Refusal (Jailbreaking): Used harmful and harmless instruction datasets from Arditi et al. to build vectors that push the model away from refusing a request.
- Evaluation: Success was measured using the
langdetectlibrary for German and the HarmBench-Llama-2-13b classifier for jailbreaking.
Key Insights and Ideas
- The Reachability Hypothesis: The paper confirms that for many concepts, internal steering is "reachable" from the input. For the German task, the discrete suffix reproduced the steering vector's behavior almost perfectly.
- Concept Asymmetry: Steering a model to speak German was significantly easier than steering it to bypass refusal. This suggests that "language" might be a simpler linear feature in the activation space than the complex logic required for "jailbreaking."
- Readability vs. Functionality: Interestingly, "translating" a gibberish suffix into readable German actually improved performance and transferability for the language task. This suggests that human-readable instructions and optimized adversarial suffixes can sometimes align in the activation space.
- Transferability: Universal suffixes for German transferred well to new prompts and even different models within the same family, whereas anti-refusal suffixes were much more model-specific.
Key Results
- German Task: ActGCG achieved success rates of 87% to 97% across models like Gemma-2 and Qwen3, vastly outperforming standard GCG (which hovered between 37% and 83%).
- Refusal Task: The results were more nuanced. While ActGCG was highly successful at avoiding "refusal phrases" (91% success), it only achieved a 27% to 41% jailbreak rate according to the stricter HarmBench classifier.
- Optimization Efficiency: The authors found that 250 steps of optimization were generally sufficient; increasing to 500 steps provided diminishing returns relative to the computational cost.
Limitations and Future Work
The authors acknowledge several limitations:
- Scale: The experiments were conducted on a relatively small scale (30-60 prompts).
- Discrete Bottleneck: There is a clear gap between "soft" (continuous) suffixes and "hard" (discrete) tokens, indicating that the limited vocabulary of LLMs is a major constraint on matching internal states.
- Steering Vector Quality: ActGCG is only as good as the steering vector it tries to imitate. If the initial vector is weak, the suffix will be too.
Future Work: The authors suggest testing more diverse concepts beyond language and refusal, exploring more robust loss functions (like those that don't amplify large deviations as much as squared L2), and investigating if suffixes can be optimized to match activations from entirely different, "better-behaved" models.
Summary updated