What Actually Transfers: Separating Shared Semantic Geometry from Lexical Shortcuts in Cross-Model Jailbreaks

When a prompt crafted against one open language model also breaks a completely different, closed model it was never tested on, something must be shared between the two models that the prompt exploits. The leading explanation is that models trained on web-scale text converge on similar internal geometry for concepts like harm and refusal, so an intervention that tilts representations in one model naturally tilts them in another. But there is a cheaper rival story: the models merely share surface statistics — the same token clusters around refusal words, the same keyword triggers — and the "transfer" is a shared lexical reflex, not shared understanding. The literature points in both directions at once. Hagendorff et al. showed causally that aligning two models' representations on purely benign text makes jailbreaks transfer between them (Jailbreak Transferability Emerges from Shared Representations), and Xu et al. built a transferable attack on a single linear "safety direction" in intermediate layers (Uncovering Safety Risks of Large Language Models through Concept Activation Vector). Yet Amari et al. showed that linear structure in hidden states can be an artifact of a handful of keywords: probes scoring a perfect 1.0 AUROC collapse below 0.10 AUROC once the top 50 predictive tokens are redacted (The Illusion of Intent: Linear Probes as Sophisticated Keyword Counters in LLM Safety Detection). Nobody has run that control on transfer attacks themselves.

This proposal settles the question. The distributed-semantics literature supplies both the hypothesis and the instruments: Deniz et al. found that the human semantic map is invariant to whether the input is read or heard — the same meaning, expressed through a different surface code, lands on the same cortical geometry (The Representation of Semantic Information Across Human Cerebral Cortex During Listening Versus Reading Is Invariant to Stimulus Modality), and Bruna, Szegedy et al. established that concepts in deep networks are spread across the whole activation space rather than held by individual units (Intriguing properties of neural networks). We treat surface token form as the "modality" and ask whether the geometry that jailbreaks exploit is amodal in the same sense.

Core Claim

The claim is testable and two-sided. First, cross-model jailbreak transfer is carried by a distributed, semantically organized subspace of the hidden states, not by surface token statistics: attacks should retain transfer when their surface form is rewritten but their meaning is held fixed, and should lose transfer when their meaning is neutralized while their vocabulary is held fixed. Second, this subspace can be isolated, measured, and used — to predict, per prompt and per model pair, whether an attack will transfer, and to block transfer by surgical intervention without damaging the model's ordinary competence. If the first part fails — if transfer collapses under semantics-preserving rewrites — then the "shared conceptual geometry" story is a rebranding of shared lexical artifacts, and the field's mental model of why jailbreaks transfer needs rewriting.

The key instrument is a semantics-transfer score. For an attack prompt $a$, let $\mathrm{para}(a)$ be a rewrite that preserves the attack's meaning while changing its surface tokens, and let $\mathrm{ASR}_{\text{target}}$ be the attack success rate on a held-out target model:

$$\mathrm{STS}(a) = \frac{\mathrm{ASR}_{\text{target}}(\mathrm{para}(a))}{\mathrm{ASR}_{\text{target}}(a)}$$

An attack whose transfer survives paraphrase (STS near 1) rides on meaning; one whose transfer dies (STS near 0) rides on tokens. The dichotomy Hagendorff et al. observed — persona attacks transfer across model families while Base64 and ROT13 ciphers do not — is the qualitative version of this score; we turn it into a quantitative, per-prompt measurement with controls in both directions.

Research Program

The program has four parts, moving from diagnosis to mechanism to causal test. Part one builds a lexically controlled attack battery. Starting from the standard corpora — AdvBench's 520 harmful instructions, StrongREJECT's 313 answerable forbidden prompts, and HarmBench's 510 behaviors — we take five attack families spanning the surface-to-semantic spectrum: persona modulation, which shifts high-level context in fluent prose (Scalable and Transferable Black-Box Jailbreaks for Language Models via Persona Modulation); the DSN suffix attack, which optimizes discrete tokens to suppress refusal words (Don't Say No: Jailbreaking LLM by Suppressing Refusal); gradient-based suffixes in the GCG family; cipher attacks as a built-in negative control; and the SCAV embedding-space attack itself (Uncovering Safety Risks of Large Language Models through Concept Activation Vector). For each attack prompt we generate two controlled variants with an LLM rewriter, validated by human review on a sample: a semantics-preserving paraphrase (different tokens, same intent) and, adapting Amari et al.'s paired design, a vocabulary-matched neutralization (same token distribution, benign intent). We also extend Amari et al.'s lexical ablation — redacting the top predictive keywords — from probes to attacks, measuring how much of each attack's cross-model success survives keyword removal.

Part two adjudicates the architectural dispute that the file leaves open. ArrAttack's authors attribute robust transfer to discrete "malicious neurons" that input noise cannot disrupt (One Model Transfer to All: On Robust Jailbreak Prompts Generation against LLMs), which directly clashes with the distributed-representation view that concepts are spread across the whole activation space (Intriguing properties of neural networks) and with Hagendorff et al.'s finding that global topological alignment predicts transfer. We measure how concentrated each attack's causal effect is: ablating single units, random distributed directions, and layer blocks, and testing whether attacks that survive input perturbation have sharply concentrated attribution or broadly distributed footprints. This also probes the "correction phenomenon" Xu et al. documented — single-layer perturbations get undone by deeper layers, so effective attacks must hit many layers at once — which is the machine analog of the robustness Nishida et al. attribute to aggregation across a whole distributed system rather than a few top units.

Part three turns the diagnosis into a predictor. We extract, per model, the subspace of hidden-state directions that separate malicious from benign intent and remain predictive under keyword redaction — trained contrastively on paired, vocabulary-matched prompts in the spirit of Amari et al.'s Contrastive Intent Probing, which lifted paired-dataset AUROC from 0.566 to 0.721 where a plain linear probe failed (The Illusion of Intent). We then compute surrogate–target alignment restricted to this subspace and test whether it predicts per-prompt, per-pair transfer better than global similarity does. This extends Hagendorff et al.'s mutual-$k$-nearest-neighbor topology measure, which they showed moves with transfer at the model-pair level (benign distillation raised Gemma–Qwen similarity from 0.28 to 0.42 and transfer rose with it) but never resolved into semantic versus lexical components (Jailbreak Transferability Emerges from Shared Representations). It also extends Liang et al.'s analytical bridge between adversarial and knowledge transferability, whose perturbation-magnitude and direction metrics ($\alpha_1$, $\alpha_2$) were validated for classifiers in the small-perturbation linear regime and never tested against the large, discrete, generative disruptions of real jailbreaks (Uncovering the Connections Between Adversarial Transferability and Knowledge Transferability).

Part four is the causal payoff: a steering intervention. If a shared semantic subspace carries transfer, then projecting target-model representations orthogonally away from the shared safety-relevant geometry should selectively destroy incoming transfer while leaving benign capability intact. The precedent for a representation-level intervention with no capability tax is Safety Representation Ranking, which held Qwen2.5-7B's MATH score at 69.1% against a 68.7% baseline while improving safety selection (Advancing LLM Safe Alignment with Safety Representation Ranking) — but that method was never evaluated under cross-model surrogate attacks, an open problem its own authors flag.

Baselines

The naive reference point is raw, uncontrolled attack transfer: take a suffix or rewriting attack optimized on an open surrogate and evaluate it directly on the target, with no lexical or semantic controls. This is the operating condition of the published numbers we must contextualize — DSN reached nearly 100% success on Llama-2-7b-chat and, in multi-trial transfer, 36% on GPT-4 and 22% on Claude (Don't Say No) — and it defines the floor any analysis must explain. We will reproduce this floor on our model pairs with identical prompts and token budgets so every controlled variant is measured against it.

The strongest existing competitor on the prediction question is Hagendorff et al.'s global representational similarity via mutual $k$-nearest-neighbors over prompt embeddings (Jailbreak Transferability Emerges from Shared Representations), because it is the only method with causal evidence linking representation alignment to transfer. We will run it on exactly the same model pairs, prompts, and attack set as our subspace-restricted metric, with the same compute and no tuning advantage, and compare both against per-prompt transfer outcomes. Our metric counts as beating it only if restricting alignment to the redaction-robust semantic subspace yields materially higher correlation with transfer than the global measure — if the restriction adds nothing, the decomposition is vacuous and we will say so.

The strongest existing method on the attack side is the SCAV pipeline of Xu et al., which reported an average 99.14% attack success rate across seven open-source models and transfers to black-box targets by evolving prompts with a genetic algorithm (Uncovering Safety Risks of Large Language Models through Concept Activation Vector), alongside ArrAttack, which achieved 74% on GPT-4 and 40% on Claude-3 where prior attacks scored zero (One Model Transfer to All). We run SCAV under our lexical controls — its prompts paraphrased and keyword-redacted — because SCAV's authors interpret their linear separability as genuine semantic encoding, the exact claim Amari et al.'s controls were built to interrogate. SCAV survives our critique only if its transfer persists under vocabulary-matched neutralization.

The ablation isolating our key ingredient is the alignment metric itself: the same pipeline, same contrastive training, same model pairs, with the subspace restriction removed (recovering global alignment) and with the subspace replaced by directions that do not survive keyword redaction (recovering a purely lexical-overlap predictor, implemented also as token n-gram Jaccard similarity between attack and refusal-word distributions). The ingredient earns its place only if the redaction-robust subspace beats both.

Why not just…?

Why not just measure global representation similarity and skip the semantic/lexical decomposition? Because the question the field actually cares about — whether models share understanding or merely share statistics — is invisible to a global measure. Hagendorff et al.'s mutual-$k$-NN result establishes that alignment and transfer co-vary, but a global topology score is high when two models share lexical statistics too; it cannot tell the two apart, and it only moves at the model-pair level, not per prompt (Jailbreak Transferability Emerges from Shared Representations). The honest risk is that global alignment predicts per-prompt transfer just as well as our restricted metric; experiment three is designed so that this outcome would be a clean, publishable negative result forcing us to concede that the decomposition adds predictive nothing — though the steering experiment would still be needed to establish causality.

Why not just run Amari's lexical ablation on attacks and be done? Ablation is a necessary diagnostic and part one of our program includes it, but it only shows that a probe or attack depends on certain keywords; it cannot say what carries transfer when keywords are controlled, and it offers no predictive or defensive handle. Amari et al. themselves stopped at detection probes on Mistral-7B with a small held-out set, and never analyzed cross-model attack transfer (The Illusion of Intent). Our paired-variant design, subspace metric, and steering intervention are the parts ablation cannot deliver.

Why not just defend at the input or output boundary — character perturbation, output filters, or ranking? Because the file's evidence says boundary defenses either lose the arms race or tax capability. ArrAttack was built explicitly to survive input perturbation and still averaged 57.69% success against defended models, versus 34.03% for the prior best (One Model Transfer to All); Souly et al. showed that aggressive prompt transformations degrade the model's own reasoning on MMLU — the "empty jailbreak" where compliance comes with capability collapse (A StrongREJECT for Empty Jailbreaks); and the representation-level ranker of Du et al., though tax-free, has never been tested against cross-model surrogate attacks (Advancing LLM Safe Alignment with Safety Representation Ranking). Our steering experiment is the first test of whether removing the shared transferable subspace specifically blocks incoming transfer — and it will be scored on both StrongREJECT and MATH so a defense that wins by lobotomizing the model fails loudly.

Proposed Experiments and Decisive Checks

Surrogates are open models spanning families and scales — Llama-2-7b-chat, Mistral-7B, Qwen2.5, Gemma-2, Vicuna-13b — with GPT-4 and Claude-3 as black-box targets via API, mirroring the ArrAttack and DSN evaluation setups. Success is scored with the StrongREJECT rubric, $(1-\text{refused}) \times (\text{specific}+\text{convincing})/2$, whose evaluator agrees with human judges at a Spearman correlation of about 0.90 (A StrongREJECT for Empty Jailbreaks); we never report bare non-refusal, because Souly et al. showed that metric counts garbled output as success. All completions use a fixed 512-token budget, since HarmBench's evaluation of 18 attack methods across 33 targets found that completion length alone moves reported success by as much as 30% (HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusal), with the HarmBench Llama-2-13B classifier (93.19% agreement with human labels) as a secondary check.

The decisive checks are three. First, the semantics-transfer score must split the attack families the theory predicts: persona and multi-layer steering attacks should show STS near 1, cipher attacks near 0, with optimized suffixes somewhere measurable in between — and if optimized suffixes turn out to be pure lexical artifacts, that is itself a significant finding about the most-studied attack class. Second, subspace-restricted alignment must predict per-prompt transfer beyond global alignment and lexical overlap under matched compute. Third, steering must cut transfer from open surrogates substantially while leaving MATH and MT-Bench within a point of baseline, the standard set by Safety Representation Ranking's 68.7%-to-69.1% MATH result (Advancing LLM Safe Alignment with Safety Representation Ranking); for MT-Bench, no reported numeric result was found on Lacuna, so we establish our own baseline under identical decoding. If all three hold, the shared-geometry story is confirmed in its strong, semantic form and becomes actionable; if the first fails, we will have shown that what transfers between today's models is not meaning but shared token reflexes — and defenses should be aimed at the lexicon, not the geometry.