NegBLEURT Forest: Leveraging Inconsistencies for Detecting Jailbreak Attacks

Lama Sleem, Jérôme François, Lujun Li, Nathan Foucher, Niccolò Gentile, Radu StatearXiv 2025 · DOI · arXiv

Abstract

Large Language Models (LLMs) are increasingly guarded by safety alignments, yet "jailbreak" attacks—cleverly engineered prompts designed to bypass these safeguards—remain a persistent threat. Current defenses often rely on fragile keyword lists or semantic embeddings that fail to distinguish between a refusal and an agreement. We introduce NegBLEURT Forest, a novel framework that detects jailbreak attacks by identifying semantic inconsistencies in model responses. By combining a negation-aware scoring metric with anomaly detection via Isolation Forests, the system reliably identifies successful attacks. Our results demonstrate that NegBLEURT Forest significantly outperforms existing baselines, achieving an F1 score of 0.899 on Llama-2-7b.

Introduction

The rapid deployment of Large Language Models (LLMs) has necessitated robust safety mechanisms to prevent the generation of harmful content. However, "jailbreaking" remains a significant challenge. Attackers use sophisticated prompting techniques to trick models into ignoring their safety training. Existing defenses typically fall into two categories: preprocessing (e.g., input smoothing) and post-processing (e.g., output filtering).

A major limitation of current post-processing methods is their reliance on "hard" thresholds or specific keyword triggers (e.g., "I cannot"). These are easily bypassed by creative phrasing. Furthermore, standard semantic embeddings are often "negation-blind"; the vectors for "I can help" and "I cannot help" are mathematically similar, making it difficult for automated systems to distinguish a refusal from a successful jailbreak.

The NegBLEURT Forest Framework

NegBLEURT Forest shifts the focus from simple keyword matching to analyzing the semantic consistency and attitudinal intent of model responses. The framework operates in four distinct stages:

  1. Prompt Perturbation: Inspired by "SmoothLLM," the system generates multiple variations of an input prompt using word insertion, patching, and swapping. This exploits the "brittleness" of jailbreaks; while a specific prompt might bypass safety, slight variations often cause the model to revert to a refusal.
  2. Salient Sentence Extraction: LLM responses often contain "chatter" that can mask intent. Using a zero-shot classifier (BART-large-mnli), the system extracts the most "attitudinal" sentence—specifically looking for refusal or apology patterns—and truncates it to an optimal length of 15-20 tokens.
  3. Negation-Aware Scoring: To overcome the negation-blindness of standard embeddings, the framework utilizes NegBLEURT. This metric is specifically designed to be sensitive to negations. The extracted sentence is compared against a Refusal Semantic Domain (RSD)—a curated "anchor" set of approximately 50 common refusal phrases.
  4. Anomaly Detection: The system treats a successful jailbreak as an outlier. An Isolation Forest algorithm analyzes the model's response in the negation-aware feature space. If a response deviates significantly from the expected refusal behavior (the RSD), it is flagged as a successful attack.
System architecture of the NegBLEURT Forest framework.

Experimental Evaluation

The framework was evaluated using the JailbreakBench and JailbreakV-28K datasets. Due to inconsistencies in existing labels, the authors created a manually validated dataset of 161 original harmful prompts. Testing was conducted on two prominent open-source models: Llama-2-7b-chat-hf and Gemma-2-9b.

The "Negation Gap"

A key finding of the study is the statistical difference between standard cosine similarity and NegBLEURT. While cosine similarity scores for successful and failed attacks often overlap, NegBLEURT creates a clear separation, allowing for more accurate classification.

Comparison of BLEURT and Cosine scores showing the 'Negation Gap'.

Key Results and Insights

NegBLEURT Forest consistently outperformed state-of-the-art (SOTA) baselines like SmoothLLM and JailGuard.

  • Performance: On Llama-2, the framework achieved an F1 score of 0.899, significantly higher than the SMLM-CLS baseline (0.171).
  • Robustness: Unlike competing methods that show sensitivity to specific types of noise (e.g., swapping vs. inserting words), NegBLEURT Forest maintained high performance across various perturbation types.
  • Generalizability: The framework ranked highly across different models without requiring extensive re-tuning, suggesting it can adapt to new LLM releases.

The research highlights that successful jailbreaks are often "brittle." Slight perturbations to a jailbreak prompt frequently result in the model "snapping back" into a refusal state. By defining refusal as a "domain" rather than a specific string, NegBLEURT Forest provides a flexible and robust defense.

Conclusion and Future Work

NegBLEURT Forest represents a significant step forward in LLM safety by leveraging semantic inconsistencies and negation-aware metrics. However, challenges remain. The computational overhead of generating multiple responses and running anomaly detection is non-trivial. Additionally, the Refusal Semantic Domain (RSD) requires periodic manual curation to maintain high-quality anchors. Future work will focus on optimizing the extraction and scoring algorithms to reduce latency, making the framework more suitable for real-time production environments.

Summary updated

Abstract

Jailbreak attacks designed to bypass safety mechanisms pose a serious threat by prompting LLMs to generate harmful or inappropriate content, despite alignment with ethical guidelines. Crafting universal filtering rules remains difficult due to their inherent dependence on specific contexts. To address these challenges without relying on threshold calibration or model fine-tuning, this work introduces a semantic consistency analysis between successful and unsuccessful responses, demonstrating that a negation-aware scoring approach captures meaningful patterns. Building on this insight, a novel detection framework called NegBLEURT Forest is proposed to evaluate the degree of alignment between outputs elicited by adversarial prompts and expected safe behaviors. It identifies anomalous responses using the Isolation Forest algorithm, enabling reliable jailbreak detection. Experimental results show that the proposed method consistently achieves top-tier performance, ranking first or second in accuracy across diverse models using the crafted dataset, while competing approaches exhibit notable sensitivity to model and data variations.
BibTeX
@misc{sleem2025negbleurt,
  author = {Lama Sleem and Jérôme François and Lujun Li and Nathan Foucher and Niccolò Gentile and Radu State},
  title = {{NegBLEURT Forest: Leveraging Inconsistencies for Detecting Jailbreak Attacks}},
  year = {2025},
  eprint = {2511.11784},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2511.11784},
}
Figure