Evaluating Multi-Agent Defences Against Jailbreaking Attacks on Large Language Models

Maria Carolina Cornelia Wit, Jun PangarXiv 2025 · DOI · arXiv

Introduction

As Large Language Models (LLMs) become increasingly capable, the challenge of "jailbreaking"—designing prompts to circumvent safety filters—has intensified. Traditional defense strategies, such as safety fine-tuning, often struggle to keep pace with the rapid evolution of attack templates. This paper investigates the use of Multi-Agent Systems (MAS) as a modular defense layer. Rather than filtering input prompts, these systems employ a pipeline of specialized agents to analyze the LLM's response. The study explores whether increasing the number of agents improves robustness or primarily leads to increased "over-blocking" and latency.

Methodology

The researchers reproduced and extended the AutoDefense framework, evaluating three distinct architectural configurations to determine the impact of agent specialization.

Architectural configurations of the AutoDefense framework: Single-Agent, Two-Agent, and Three-Agent setups.

The configurations tested include:

  1. Single-Agent: A standalone LLM classifies the response as VALID or INVALID.
  2. Two-Agent: An Analyzer infers the user's intent, while a Judge makes the final safety determination.
  3. Three-Agent: Analysis is split between an Intention Analyzer and a Prompt Analyzer, followed by a final verdict from a Judge.

A Coordinator agent manages the communication between these components. To evaluate performance, the authors used AttackEval (powered by GPT-4) to establish a "pseudo-ground truth" for defense accuracy.

Datasets and Metrics

The defense was tested against three jailbreak templates: the Original Attack, the BetterDan ("Do Anything Now") prompt, and a sophisticated JB (Jailbreak) template that uses a dual-layered approach to confuse filters.

Due to class imbalances in datasets like BetterDan, the authors utilized the P4 score. This symmetric metric provides a more reliable assessment of classifier performance than the standard F1-score, as it remains invariant even if positive and negative labels are swapped.

Key Insights and Results

The study identified a significant Sensitivity-Specificity Trade-off. Adding more agents consistently reduced False Negatives (FN)—meaning the system caught more malicious responses. However, it also led to a rise in False Positives (FP), where benign content was incorrectly flagged as unsafe.

For the Original Attack, the 3-agent setup was highly effective, reducing False Negatives from 27 to just 2. However, the complex JB template caused a massive spike in False Positives in the 3-agent setup (increasing from 6 to 139), as the multi-layered nature of the attack confused the specialized agents. Interestingly, in some borderline cases, the 3-agent setup provided a "balancing effect," correcting errors made by simpler configurations through more nuanced analysis.

Conclusion

While multi-agent defenses significantly enhance the ability to detect jailbreak attempts, they introduce substantial trade-offs. The added complexity results in approximately two seconds of additional latency per interaction, which may be prohibitive for real-time applications. Furthermore, the tendency toward over-blocking suggests that future work should explore hybrid strategies, including human-in-the-loop oversight and the inclusion of non-Western models to mitigate ethical and cultural biases in safety assessments.

Summary updated

BibTeX
@misc{wit2025evaluating,
  author = {Maria Carolina Cornelia Wit and Jun Pang},
  title = {{Evaluating Multi-Agent Defences Against Jailbreaking Attacks on Large Language Models}},
  year = {2025},
  eprint = {2506.23576},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2506.23576},
}
Figure