TL;DR
Test-Time Safety Alignment (TSA) neutralizes jailbreaks and harmful prompts by optimizing an LLM's input embeddings at inference time using zeroth-order gradients from a black-box moderation API. Remarkably, these continuous adjustments are "sub-lexical"—they steer the model to generate safe responses without changing the human-readable text of the prompt or introducing a blanket refusal bias.
What Problem Does This Paper Address?
Large language models undergo extensive safety alignment (such as RLHF or DPO) during training, yet adversarial "jailbreak" prompts can still exploit gaps in this alignment to elicit harmful content.
Existing test-time defenses typically manipulate discrete text tokens (which can be brittle and easily bypassed), require training separate auxiliary guard models, or apply static internal activation steering. This paper addresses the lack of a direct, continuous optimization method at the input level that can dynamically adapt to any prompt at test time without altering the underlying model's weights.
Methods
The authors introduce Test-Time Safety Alignment (TSA). Instead of modifying the text tokens of a prompt, TSA optimizes the continuous prompt embedding matrix that the model actually conditions on.
Because both the target LLM and the safety evaluator (the OpenAI Moderation API) are treated as black boxes, standard backpropagation is impossible. To get around this, TSA uses a zeroth-order gradient estimator:
- It applies random Gaussian noise to the prompt embeddings to create multiple perturbed copies.
- It runs forward passes through the LLM and queries the moderation API to see how the safety scores change.
- It uses these differences to estimate a gradient direction and updates the embeddings via gradient descent.
To prevent the embeddings from drifting into regions of latent space the model hasn't seen during training, TSA incorporates two simple regularization mechanisms: gradient normalization (ensuring consistent step sizes) and a cosine similarity constraint (projecting the embeddings back if they stray too far from the original prompt).
Datasets
The method was evaluated on two prominent red-teaming benchmarks:
- WildJailbreak: The authors utilized the adversarial harmful split (2,000 prompts rewritten with stealthy, in-the-wild jailbreak tactics) and the adversarial benign split (210 prompts that mimic jailbreak structures but have harmless intent, serving as a control).
- HarmBench: A dataset of 400 direct, explicit harmful queries violating standard AI safety policies.
Key Insights and Ideas
- Sub-Lexical Control: Surprisingly, the optimized embeddings still decode back to the exact same original text tokens. The perturbation is small enough to stay within each token's Voronoi cell, meaning the human-readable prompt remains identical, yet the subtle continuous shift completely redirects the LLM's internal hidden states toward safe generation.
- Asymmetry of Safety Landscapes: The authors attempted to reverse the optimization to generate jailbreaks, but it failed. Aligned models sit within a broad "refusal basin" where local perturbations continue to trigger refusals. The narrow compliance pathways exploited by sophisticated jailbreaks lie beyond the reach of local isotropic probes.
- No Blanket Refusal Bias: TSA employs early stopping (halting optimization once the moderation score drops below 0.1). Because it only intervenes when a prompt actually triggers a safety risk, it leaves benign prompts entirely unaffected, avoiding the over-refusal tendencies of other defenses.
Key Results
- Complete Neutralization: TSA completely eliminated safety-flagged responses (reducing the flagged count to 0) for 4 out of 5 tested models (GPT-OSS-20B, Qwen3-14B, Llama 3.1-8B, and Phi-3.5-4B) across both HarmBench and WildJailbreak.
- Rapid Convergence: For well-aligned models like Llama 3.1-8B, the optimization typically converged in a single gradient step.
- Gemma 3 Exception: Gemma 3-1B was the sole exception, failing to resolve a fraction of harmful prompts. The authors hypothesize this is due to its smaller embedding dimension (1152) or thinner baseline safety training.
- Preserving Benign Inputs: On the adversarial benign control split, already-safe responses were left substantively intact, outperforming baseline defenses like SmoothLLM, AdaSteer, and RESTA, which occasionally caused false positives or increased moderation scores.
The complete breakdown of these results across models and baselines can be seen in Table 1:
Limitations and Future Work
- Per-Prompt Latency: The optimization loop introduces an overhead of several seconds per prompt. This is primarily driven by network round-trips to the cloud-based OpenAI Moderation API. Utilizing a local moderation classifier could drastically reduce this latency.
- Embedding Access Requirement: TSA requires access to the input word embedding matrix. While this condition is easily met for open-weight models, it cannot be applied to purely API-gated models (like GPT-4 or Claude) where users can only pass discrete text.
- Dependence on Base Alignment: TSA amplifies existing safety alignment rather than replacing it entirely; if a model's baseline safety training is too thin (as seen with Gemma 3), the optimizer struggles to find a safe direction within the cosine ball.
- Future Directions: A natural next step is exploring whether sub-lexical embedding optimization can be generalized to steer other test-time behaviors, such as strict instruction-following or complex policy compliance.
Summary updated