Test-Time Detoxification without Training or Learning Anything

Baturay Saglam, Dionysis Kalogerias

Large language models (LLMs) are trained on massive web crawls, meaning they inevitably absorb the darker corners of the internet—including hate speech, bias, and toxic language. Even when provided with a perfectly benign prompt, a model can unexpectedly veer into inappropriate territory.

Mitigating this toxicity is critical for deploying safe, user-trusted AI systems at scale. However, existing solutions typically fall into three restrictive categories:

  1. Retraining or Fine-tuning: This is computationally expensive and must be repeated whenever the model updates or safety guidelines change.
  2. Decoding-time Control: These methods require access to the model's internal probability distributions (logits), which is often impossible with black-box APIs.
  3. Auxiliary Models: Relying on a secondary "safety" model to guide the primary one adds architectural complexity and latency.

To bridge this gap, researchers have introduced TIDE (Test-time Iterative Detoxification via Embeddings). TIDE is a model-agnostic, training-free method designed to detoxify any model—even those accessible only via forward passes—without sacrificing the creative quality and fluency of the generated text.

Introducing TIDE: Steering via Embeddings

TIDE treats the entire LLM pipeline as a black box, relying on a core engine of zeroth-order optimization.

In traditional white-box settings, developers calculate a gradient—the mathematical "slope" that dictates how to adjust inputs to lower toxicity. Because the toxicity scoring function is often a black-box API itself, TIDE approximates this gradient instead. The process unfolds in four steps:

  • Perturbation: The system takes the input prompt's embeddings and "pokes" them in several random directions using Gaussian noise.
  • Evaluation: These slightly altered inputs are run through the model and evaluated by a toxicity scorer (such as Perspective API).
  • Gradient Estimation: By observing which random perturbations resulted in less toxic outputs, TIDE estimates a descent direction.
  • Iterative Steering: The system takes a small step in that safer direction and repeats the process for a few iterations (typically fewer than four).

Crucially, these adjustments are sub-lexical. To a human reader, the actual tokens (words) of the prompt remain identical. However, the underlying mathematical vectors representing those words are shifted just enough to steer the model's internal state toward safer regions of its latent space.

Experimental Setup

To validate TIDE, the authors evaluated the method across three major benchmarks designed to elicit or measure harmful content:

  • RealToxicityPrompts (RTP): Specifically the "challenging" subset, notorious for triggering toxic completions in most models.
  • BOLD: A dataset used to measure social biases across various domains.
  • AttaQ: A dataset focused on adversarial questions designed to bypass safety filters.

The framework was tested on a diverse array of models, including GPT-2 Large, Gemma 2-2B, Qwen3-4B, and Llama 3.1-8B.

Key Findings and Results

The empirical results demonstrate that word embeddings function as incredibly effective "steering wheels." You do not need to change the words in a prompt to fundamentally alter a model's behavior; shifting their continuous vector values is sufficient.

  • Near-Zero Toxicity: The impact on toxicity was dramatic. On the GPT-2 model using the challenging RTP benchmark, TIDE reduced the "Toxic Rate" (the percentage of completions above a strict toxicity threshold) from a baseline of 63.9% down to a staggering 0.3%.
  • Superior Trade-offs: TIDE consistently achieved a better balance between toxicity reduction and fluency (measured by perplexity) than popular baselines like RAD or SASA. Unlike methods that warp the model's output distribution—often leading to repetitive or nonsensical text—TIDE keeps the model’s native decoding logic intact by simply changing the "starting point" in the embedding space.
  • The "Good Enough" Gradient: While zeroth-order optimization theoretically requires many samples to be accurate, the authors found that a very small number of samples (as low as 8 or 16) provided a gradient estimate "good enough" to find a non-toxic path.
  • Efficiency: Thanks to early stopping mechanisms, the average number of iterations required was only 1.6 to 3.1. This added a modest overhead of about 9 seconds per prompt—a highly reasonable trade-off for high-stakes safety deployments.

Limitations and Future Directions

While TIDE offers a powerful new tool for AI safety, it does come with trade-offs. Because it requires multiple forward passes per iteration, it is inherently slower than "one-shot" detoxification methods. The authors view this as a mild limitation, particularly for server-side deployments where safety is the paramount concern.

Additionally, TIDE faces a "temperature tax." At higher sampling temperatures—where the model is instructed to be more creative and random—the toxicity objective becomes noisier. In these settings, TIDE requires more samples, and thus more compute time, to maintain its effectiveness.

Looking forward, the implications of this research extend far beyond toxicity. The authors note that this zeroth-order framework could theoretically be applied to any "proxy objective." Without ever needing to fine-tune a model's weights, developers could use this same embedding-steering technique to guide models to be more concise, adopt a specific persona, or adhere strictly to formal formatting.

Summary updated

Abstract

Large language models can produce toxic or inappropriate text even for benign inputs, creating risks when deployed at scale. Detoxification is therefore important for safety and user trust, particularly when we want to reduce harmful content without sacrificing the model’s generation quality. Many existing approaches rely on model retraining, gradients, or learned auxiliary components, which can be costly and may not transfer across model families or to truly black-box settings. We introduce a test-time procedure that approximates the gradient of completion toxicity with respect to the input embeddings and uses a small number of descent steps to steer generation toward less toxic continuations. This is achieved with zeroth-order optimization that requires only access to input embeddings, a toxicity scoring function, and forward evaluations of the model. Empirically, the approach delivers robust toxicity reductions across models and prompts and, in most settings, achieves the best overall toxicity–quality trade-off. More broadly, our work positions word embeddings as effective control variables and encourages wider use of black-box optimization to guide autoregressive language models toward scalable, safer text generation, without requiring any training or access to intermediate computations.
BibTeX
@inproceedings{saglam2026test,
  author = {Baturay Saglam and Dionysis Kalogerias},
  title = {{Test-Time Detoxification without Training or Learning Anything}},
  booktitle = {International Conference on Machine Learning},
  year = {2026},
  url = {https://openreview.net/forum?id=eiNZbsYGJv},
}
Figure