TL;DR
Debate-to-Deceive (D2D) is a novel adversarial framework that tricks Large Language Model (LLM) fact-checkers by simulating a structured, game-theoretic human debate. By evolving rhetorical strategies like logic (Logos), emotion (Pathos), and credibility (Ethos) through multi-agent competition, D2D generates highly persuasive, semantically coherent adversarial passages that bypass most modern AI defenses.
What Problem Does This Paper Address?
As LLMs are increasingly deployed for high-stakes fact-checking in professional and social domains, ensuring their robustness against manipulation is critical. Existing "knowledge attacks" generally fall into three categories, each with significant flaws:
- Gradient-based suffix attacks: These generate meaningless character sequences that are easily flagged by simple perplexity filters.
- Manual/Template-based attacks: These rely on a narrow set of fixed prompts, making them brittle and easy to detect.
- Single-agent iterative attacks: These often suffer from confirmation bias and are computationally expensive due to trial-and-error.
The authors identify a need for an attack that is semantically coherent (looks like natural language), rhetorically effective (actually persuasive), and robust across diverse defense mechanisms.
Methods
The D2D framework formalizes adversarial generation as an evolutionary game played between two agents: one arguing for the true fact and another arguing for a "flipped" (false) target label.
Key components include:
- Structured Debate: D2D follows a four-phase "Singapore format" debate: Foundational reasoning (definitions), Associative reasoning (standard arguments), Core reasoning (novel insights/rebuttals), and Conclusive reasoning (synthesis).
- Hierarchical Strategy Layer: To avoid the intractably large space of all possible text outputs, the authors decouple agent behavior. A strategic layer selects a rhetorical stance (Logos, Pathos, or Ethos) using Multi-Agent Reinforcement Learning (MARL), while a generation layer produces the actual text conditioned on that strategy.
- Game-Theoretic Payoff: The interaction is modeled as a symmetric zero-sum game. The "payoff" for a strategy depends on how well it counters the opponent's strategy, governed by a skew-symmetric matrix.
- Dual-Reward System: The agents are optimized using a macro-level strategic reward (game gains) and a micro-level execution reward (text quality and computational cost).
Datasets
The authors evaluated D2D across four diverse fact-checking benchmarks:
- FEVER: General-domain factual claims.
- HealthVer: Biomedical and public health claims.
- CLIMATE-FEVER: Climate science claims.
- SciFact: Scientific claim verification.
- SimpleFacts: Used as a control set for benign evaluation to ensure defenses don't break normal model utility.
Key Insights and Ideas
- Persuasion as a Weapon: Unlike "jailbreaks" that try to find a magic string to break safety filters, D2D treats deception as a strategic, human-like process. It wins by being more persuasive than the truth.
- Multi-Agent Synergy: The competitive nature of the debate prevents the "confirmation bias" seen in single-agent methods. The agents essentially "sharpen" each other's arguments, leading to more robust attacks.
- Cyclic Strategy Dynamics: The authors found that rhetorical strategies don't settle on a single "best" approach. Instead, they orbit a mixed equilibrium, ensuring the attack remains diverse and unpredictable for defenders.
- Efficiency through Decoupling: By optimizing in a compact strategy space (Logos/Pathos/Ethos) rather than raw token space, D2D is significantly faster and requires less memory than gradient-based or optimization-heavy methods.
Key Results
D2D demonstrated state-of-the-art performance across multiple target models (Llama3, Gemma2, Mistral, and Qwen3).
- High Success Rates: Without defenses, D2D achieved a 98.8% average Attack Success Rate (ASR) across all datasets when using Qwen3-32B.
- Defense Evasion: D2D outperformed all baselines under various defense conditions, including instructional, LLM-based, and response-based detection. Notably, it maintained high ASR (up to 100% in some cells) where baselines like AutoDAN collapsed to 0%.
- Speed: D2D is an order of magnitude faster than optimization-based baselines like JudgeDeceiver and ADMIT, making it a more practical threat.
Limitations and Future Work
- Alignment Constraints: The authors noted that model alignment still acts as a barrier; agents sometimes refuse to argue for blatant health falsehoods, which limits the attack's reach in sensitive domains.
- Defense Development: While this paper focuses on the attack, the authors emphasize that understanding these strategic vulnerabilities is the first step toward building game-theoretic defenses that can withstand adaptive adversaries.
- Ethical Guardrails: Due to the potential for misuse in misinformation campaigns, the authors have chosen not to release the full code or prompt templates, providing instead the methodological framework for research purposes.
Summary updated