RvB: Automating AI System Hardening via Iterative Red-Blue Games

Lige Huang, Zicheng Liu, Jie Zhang, Lewen Yan, Dongrui Liu, Jing ShaoarXiv 2026 · arXiv · OpenReview

TL;DR

This paper introduces the Red Team vs. Blue Team (RvB) framework, a novel approach to automatically hardening AI systems by simulating an iterative game between an attacker (Red Team) and a defender (Blue Team). RvB drives continuous improvement in AI security by forcing the defender to learn robust defenses against evolving threats without requiring model retraining, achieving impressive success rates in code hardening and content security.

What Problem Does This Paper Address?

Large Language Models (LLMs) are a double-edged sword in AI security: they can be used to find vulnerabilities and launch attacks, but also to defend systems and patch exploits. However, current research in AI offense and defense often operates in silos. Offensive tools lack a responsive adversary to test their true resilience, while defensive methods often rely on static benchmarks that can be easily bypassed by novel attack vectors. This creates a critical gap: a lack of unified frameworks for dynamic, iterative adversarial adaptation that can continuously harden AI systems against emerging threats.

Methods

The core of the RvB framework is a training-free, sequential, imperfect-information game. It models AI hardening as an iterative, turn-based interaction between a Red Team (attacker) and a Blue Team (defender).

  • Red Team: Probes the system to find vulnerabilities and generates "attack logs" detailing the exploits.
  • Blue Team: Uses these attack logs to learn and synthesize defensive solutions (e.g., code patches, guardrail rules) without updating its own model parameters.
  • Iterative Process: The Blue Team's defenses update the system's state, creating a new environment for the Red Team to attack in the next round. This cycle forces the Blue Team to adapt and learn more generalizable defensive principles.
  • Imperfect Information: The Blue Team doesn't know the Red Team's exact strategy, and vice-versa, leading to a more realistic adversarial dynamic.
  • Externalized Memory: The environment itself acts as an externalized memory, storing the history of interactions and driving the state transitions. This avoids the need for complex internal state management within the agents.

Datasets

The paper validates RvB on two challenging domains:

  • Cyber Security (Code Hardening): They used 10 specific vulnerabilities, including CVE-2022-30887, within a Pharmacy Management System v1.0, simulated in a PHP service environment.
  • Content Security (Guardrail Optimization): They utilized harmful prompts from the HarmBench benchmark (Mazeika et al., 2024) as attack targets.

Key Insights and Ideas

  • Adversarial Feedback as a Curriculum: The iterative interaction acts as a powerful, unsupervised curriculum. The Red Team's persistent probing for weaknesses forces the Blue Team to move beyond superficial fixes and learn fundamental, robust defensive principles.
  • Generalization without Overfitting: The framework successfully avoids catastrophic forgetting. The Blue Team learns generalizable rules that defend against both new and older attack patterns, and importantly, does so without significantly increasing false positives.
  • Semantic Verification for Safety: The Red Team's immediate exploitation of any newly introduced vulnerabilities acts as a semantic verifier for the Blue Team's patches. This prevents the common issue in automated repair where fixes might break functionality or introduce new problems.
  • Efficiency Through Precision: RvB is more computationally efficient than traditional cooperative MAS. The Red Team's targeted attack logs prune the Blue Team's search space, reducing redundant reasoning and token consumption.

Key Results

  • Cyber Security:
    • Achieved a 90% Defense Success Rate (DSR) in dynamic code hardening.
  • Maintained a near-zero False Positive Rate (FPR).
  • Significantly surpassed a cooperative MAS baseline in DSR, improving from a lower initial rate to 90% by iteration 5.
  • Demonstrated a Service Disruption Rate (SDR) of 0%, compared to 60% for the baseline, indicating robust and non-disruptive fixes.
  • Content Security:
    • Achieved a 45% DSR in guardrail optimization.
  • Showed a steady increase in DSR from 0% to 45% over iterations.
  • The Average Attack Turns (AAT) required for a successful jailbreak increased by over 7-fold, indicating progressively stronger guardrails.
  • Demonstrated strong out-of-domain generalization, with the final guardrail achieving 99% and 100% defense success rates on unseen jailbreak benchmarks.

Limitations and Future Work

  • Limitations:
    • The current experiments used a representative but constrained set of web vulnerabilities. Further testing on larger, more diverse benchmarks is needed to verify generalizability.
  • The turn-based simulation is a controlled abstraction and simplifies the asynchronous dynamics of real-world security operations.
  • The efficacy of the framework relies on the reasoning capabilities of the underlying foundation models.
  • Future Work:
    • Explore theoretical bounds of convergence in multi-modal environments.
  • Investigate the application of RvB to more complex, long-horizon security tasks.

Summary updated

Abstract

The dual offensive and defensive utility of Large Language Models (LLMs) highlights a critical gap in AI security: the lack of unified frameworks for dynamic, iterative adversarial adaptation hardening. To bridge this gap, we propose the Red Team vs. Blue Team (RvB) framework, formulated as a training-free, sequential, imperfect-information game. In this process, the Red Team exposes vulnerabilities, driving the Blue Team to learning effective solutions without parameter updates. We validate our framework across two challenging domains: dynamic code hardening against CVEs and guardrail optimization against jailbreaks. Our empirical results show that this interaction compels the Blue Team to learn fundamental defensive principles, leading to robust remediations that are not merely overfitted to specific exploits. RvB achieves Defense Success Rates of 90\% and 45\% across the respective tasks while maintaining near 0\% False Positive Rates, significantly surpassing baselines. This work establishes the iterative adversarial interaction framework as a practical paradigm that automates the continuous hardening of AI systems.
BibTeX
@misc{huang2026rvb,
  author = {Lige Huang and Zicheng Liu and Jie Zhang and Lewen Yan and Dongrui Liu and Jing Shao},
  title = {{RvB: Automating AI System Hardening via Iterative Red-Blue Games}},
  year = {2026},
  eprint = {2601.19726},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2601.19726},
}
Figure