Debate to Deceive: Poisoning LLM's Fact-Checking via Game-Theoretic Multi-Agent Debate

Liting Huang, Huizhi Liang, Yi Lei, Xingjun Ma, Varun OjhaarXiv 2026 · arXiv

TL;DR

Debate-to-Deceive (D2D) is a novel adversarial framework that tricks Large Language Model (LLM) fact-checkers by simulating a structured, game-theoretic human debate. By evolving rhetorical strategies like logic (Logos), emotion (Pathos), and credibility (Ethos) through multi-agent competition, D2D generates highly persuasive, semantically coherent adversarial passages that bypass most modern AI defenses.

What Problem Does This Paper Address?

As LLMs are increasingly deployed for high-stakes fact-checking in professional and social domains, ensuring their robustness against manipulation is critical. Existing "knowledge attacks" generally fall into three categories, each with significant flaws:

  1. Gradient-based suffix attacks: These generate meaningless character sequences that are easily flagged by simple perplexity filters.
  2. Manual/Template-based attacks: These rely on a narrow set of fixed prompts, making them brittle and easy to detect.
  3. Single-agent iterative attacks: These often suffer from confirmation bias and are computationally expensive due to trial-and-error.

The authors identify a need for an attack that is semantically coherent (looks like natural language), rhetorically effective (actually persuasive), and robust across diverse defense mechanisms.

Methods

The D2D framework formalizes adversarial generation as an evolutionary game played between two agents: one arguing for the true fact and another arguing for a "flipped" (false) target label.

The D2D framework orchestrates two agents in a structured debate to mislead a target LLM.

Key components include:

  • Structured Debate: D2D follows a four-phase "Singapore format" debate: Foundational reasoning (definitions), Associative reasoning (standard arguments), Core reasoning (novel insights/rebuttals), and Conclusive reasoning (synthesis).
  • Hierarchical Strategy Layer: To avoid the intractably large space of all possible text outputs, the authors decouple agent behavior. A strategic layer selects a rhetorical stance (Logos, Pathos, or Ethos) using Multi-Agent Reinforcement Learning (MARL), while a generation layer produces the actual text conditioned on that strategy.
  • Game-Theoretic Payoff: The interaction is modeled as a symmetric zero-sum game. The "payoff" for a strategy depends on how well it counters the opponent's strategy, governed by a skew-symmetric matrix.
The payoff matrix defines the relative advantages between Logos, Pathos, and Ethos.
  • Dual-Reward System: The agents are optimized using a macro-level strategic reward (game gains) and a micro-level execution reward (text quality and computational cost).

Datasets

The authors evaluated D2D across four diverse fact-checking benchmarks:

  • FEVER: General-domain factual claims.
  • HealthVer: Biomedical and public health claims.
  • CLIMATE-FEVER: Climate science claims.
  • SciFact: Scientific claim verification.
  • SimpleFacts: Used as a control set for benign evaluation to ensure defenses don't break normal model utility.

Key Insights and Ideas

  • Persuasion as a Weapon: Unlike "jailbreaks" that try to find a magic string to break safety filters, D2D treats deception as a strategic, human-like process. It wins by being more persuasive than the truth.
  • Multi-Agent Synergy: The competitive nature of the debate prevents the "confirmation bias" seen in single-agent methods. The agents essentially "sharpen" each other's arguments, leading to more robust attacks.
  • Cyclic Strategy Dynamics: The authors found that rhetorical strategies don't settle on a single "best" approach. Instead, they orbit a mixed equilibrium, ensuring the attack remains diverse and unpredictable for defenders.
  • Efficiency through Decoupling: By optimizing in a compact strategy space (Logos/Pathos/Ethos) rather than raw token space, D2D is significantly faster and requires less memory than gradient-based or optimization-heavy methods.

Key Results

D2D demonstrated state-of-the-art performance across multiple target models (Llama3, Gemma2, Mistral, and Qwen3).

  • High Success Rates: Without defenses, D2D achieved a 98.8% average Attack Success Rate (ASR) across all datasets when using Qwen3-32B.
ASR results for Qwen3-32B across different datasets and defense settings.
  • Defense Evasion: D2D outperformed all baselines under various defense conditions, including instructional, LLM-based, and response-based detection. Notably, it maintained high ASR (up to 100% in some cells) where baselines like AutoDAN collapsed to 0%.
Comparison of D2D against baselines across different models and defenses.
  • Speed: D2D is an order of magnitude faster than optimization-based baselines like JudgeDeceiver and ADMIT, making it a more practical threat.

Limitations and Future Work

  • Alignment Constraints: The authors noted that model alignment still acts as a barrier; agents sometimes refuse to argue for blatant health falsehoods, which limits the attack's reach in sensitive domains.
  • Defense Development: While this paper focuses on the attack, the authors emphasize that understanding these strategic vulnerabilities is the first step toward building game-theoretic defenses that can withstand adaptive adversaries.
  • Ethical Guardrails: Due to the potential for misuse in misinformation campaigns, the authors have chosen not to release the full code or prompt templates, providing instead the methodological framework for research purposes.

Summary updated

Abstract

Knowledge attacks on Large Language Models, such as manipulating fact-checking judgments, pose significant risks to AI reliability. Existing approaches -- adversarial suffix/template injection and single-agent self-reflection -- lack robustness across diverse defenses and are prone to confirmation bias and costly trial-and-error, respectively. We propose Debate-to-Deceive (D2D), a multi-agent debate framework grounded in human debate structure and rhetorical style, enabling agents to generate adversarial passages that are semantically coherent, rhetorically effective, and resistant to early confirmation bias. D2D formalizes the adversarial process as an evolutionary game in which agents adaptively evolve their rhetorical strategies (Logos, Pathos, Ethos) through competition. To manage the intractably large LLM output space, we decouple agent behavior into a strategic layer, which selects a rhetorical action from a compact strategy space via evolutionary-inspired learning dynamics, and a generation layer, where the LLM produces a concrete passage conditioned on that strategy and the debate context. On four fact-checking benchmarks (FEVER, HealthVer, CLIMATE-FEVER, and SciFact), D2D substantially outperforms existing attacks and evades most defenses.
BibTeX
@misc{huang2026debate,
  author = {Liting Huang and Huizhi Liang and Yi Lei and Xingjun Ma and Varun Ojha},
  title = {{Debate to Deceive: Poisoning LLM's Fact-Checking via Game-Theoretic Multi-Agent Debate}},
  year = {2026},
  eprint = {2611.08756},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.08756},
}
Figure