Introduction
Large Language Models (LLMs) are increasingly integrated into various applications, yet they remain vulnerable to generating harmful content. DIESEL (Dynamic Inference-Guidance via Evasion of Semantic Embeddings) introduces a lightweight, training-free inference-guidance technique designed to steer LLMs away from unsafe outputs. By reranking tokens based on their semantic similarity to "negative concepts" in real-time, DIESEL significantly reduces the success rate of adversarial jailbreaks with minimal computational overhead.
The Challenge of LLM Safety
While techniques like Reinforcement Learning from Human Feedback (RLHF) align models with human values, they are often brittle. Adversarial "jailbreaks"—carefully crafted prompts designed to bypass safety filters—can still elicit prohibited content. Existing defenses typically fall into two categories: reactive post-hoc filters that check the full response after generation, and proactive inference-guidance methods that can increase latency by over 20x.
DIESEL addresses this trade-off by providing a proactive, lightweight safety layer that can be integrated into any autoregressive LLM without retraining, maintaining high utility while effectively blocking unsafe outputs.
Methodology
DIESEL operates during the decoding phase of an LLM, following a three-step process for every token generated:
- Candidate Selection: To maintain efficiency, DIESEL uses top-k sampling to select a small pool of the most likely next tokens (e.g., $k=5$).
- Latent Space Semantic Similarity: The current sequence and each candidate token are passed through a compact, 24M-parameter external embedding model. The system calculates the cosine similarity between these embeddings and a precomputed set of "negative concepts" such as "violence" or "hate speech."
- Token Reranking: A safety score ($\gamma$) is calculated for each candidate. If a token increases the semantic similarity to a negative concept, its probability is reduced. The model then selects the highest-scoring "safe" token.
Additionally, DIESEL features a global rejection mechanism: if the safety scores of all top candidates fall below a threshold ($\tau$), the model halts and issues a standard refusal.
Key Insights
- Efficiency of Small Models: A tiny 24M-parameter embedding model (0.34% the size of a 7B LLM) is sufficient to guide much larger models effectively.
- Semantic Robustness: By operating in the latent space rather than relying on keyword filters, DIESEL understands the intent of a sentence, making it resilient to adversarial suffixes.
- Cross-Lingual Generalization: English-language negative concepts successfully filter outputs in other languages, suggesting that the embedding space captures universal semantic concepts.
Evaluation and Results
The researchers evaluated DIESEL using datasets like AdvBench and BeaverTails for safety, and MMLU and TruthfulQA for utility.
- Robustness: On Llama 3, DIESEL reduced the success rate of adaptive jailbreak attacks from 93% to 22%.
- Performance: DIESEL adds only 1.27x to 2x latency, compared to the 20x overhead of previous methods like RAIN.
- Utility: Evaluation showed virtually zero impact on the model's performance for safe, factual prompts.
Limitations and Future Work
DIESEL's token-by-token decision-making can be improved by incorporating beam search to anticipate unsafe trajectories earlier. Future research will also focus on defining negative concepts for more abstract categories like misinformation and testing the system's performance in multi-turn dialogues.
Summary updated