CAP-CoT: Cycle Adversarial Prompt for Improving Chain of Thoughts in LLM Reasoning

Shuxu Chen, Yitian Zhou, Jiaquan Zhang, Haoyu Bian, Wenrui Hu, Aming Wu, Sungyoung Lee, Chaoning Zhang, Hyundong ShinarXiv 2026 · arXiv

TL;DR

CAP-CoT introduces a cycle-based adversarial prompt optimization framework that significantly boosts both the accuracy and stability of Large Language Model (LLM) reasoning. By pairing a solver with an evolving adversarial challenger that generates plausible but flawed reasoning chains, it iteratively refines prompts to eliminate logical vulnerabilities without adding any computational overhead at inference time.

What Problem Does This Paper Address?

Chain-of-Thought (CoT) prompting has revolutionized how LLMs handle complex tasks by breaking them down into step-by-step solutions. However, CoT reasoning remains notoriously fragile. Minor prompt perturbations, reordered premises, or irrelevant context can cause a model's reasoning chain to derail, leading to inconsistent answers across different runs.

Most existing solutions try to fix this by optimizing the forward reasoning path (e.g., prompt engineering) or using heavy test-time compute like voting (Self-Consistency) or tree searches (Tree-of-Thoughts). These methods either miss the latent failure modes the model is prone to making, or they are too computationally expensive for practical deployment. CAP-CoT addresses this gap by leveraging contrastive learning—teaching the model what not to do by explicitly contrasting correct reasoning with highly targeted, plausible mistakes.

Methods

CAP-CoT sets up an offline optimization loop consisting of three distinct roles, all powered by the same underlying LLM backbone:

  1. The Solver ($G_S$): Generates the standard forward reasoning chain for a given query.
  2. The Adversarial Challenger ($G_C$): Deliberately constructs a plausible but logically flawed "counter-chain." To bootstrap this process, the challenger uses a lightweight cold-start error taxonomy consisting of four error types: jump (skipping a step), confusion (substituting concepts/quantities), fuzzy (using vague language to hide a gap), and wrapper (embedding a wrong step in fluent text).
  3. The Feedback Agent ($F$): Compares the solver's chain and the challenger's flawed chain side-by-side, identifying exactly where the logic diverges.
Figure 1: Overview of the proposed framework CAP-CoT.

Crucially, the Feedback Agent drives a Bidirectional Structured Feedback Prompt Refinement (SFPR) process. It updates the Solver's prompt with safeguards against the exposed logical vulnerabilities, while simultaneously updating the Challenger's prompt to generate even more deceptive, targeted attacks in the next cycle. After 2 to 3 cycles, the refined Solver prompt is locked in, and the Challenger and Feedback agents are discarded. At inference time, the model operates as a standard, single-pass CoT solver.

Datasets

The authors evaluated CAP-CoT across six diverse question-answering and reasoning benchmarks:

  • MATH & GSM8K: Advanced and grade-school mathematics.
  • BBH (BIG-bench Hard): Algorithmic and logical reasoning tasks.
  • MMLU-CF: A contamination-free variant of MMLU for multi-task language understanding.
  • HotpotQA: Multi-hop question answering.
  • LongBench: Long-context understanding and reasoning.

Key Insights and Ideas

  • Task-Semantic Adversarial Attacks: Unlike safety-oriented adversarial prompting (like jailbreaks), the "adversary" here operates entirely within the semantic space of the task to uncover subtle logical blind spots.
  • The Power of Contrast: Providing the feedback agent with both a positive and a negative reasoning chain yields a much richer learning signal than critiquing a single forward chain in isolation.
  • Dynamic Error Evolution: A static set of negative examples quickly loses utility as the solver improves. By allowing the challenger to evolve its attack strategy based on the solver's current prompt, the framework ensures the negative examples remain challenging and informative.
  • Zero Inference Overhead: Because the optimization happens entirely during the prompt-design phase, the final deployed model retains the speed and low token cost of standard CoT.

Key Results

CAP-CoT was tested using four different backbones: GPT-4o-mini, Qwen-turbo, DeepSeek-V3, and GPT-4o.

  • Accuracy Gains: CAP-CoT consistently outperformed standard CoT and advanced test-time search methods (like ToT and GoT). For example, on the difficult MATH benchmark using GPT-4o, CAP-CoT achieved 89.2% accuracy, up from 79.5% with standard CoT.
Figure 2: Accuracy results (%) of our framework over optimization rounds on four LLM backbones.
  • Enhanced Stability: The framework dramatically reduced variance across runs. Even when the LLM's decoding temperature was turned up (introducing more sampling noise), the mean variation of the CAP-CoT optimized solver dropped below the 1.0 threshold by round 3, proving its robustness against perturbations.
Figure 3: Effect of LLM temperature on reasoning stability on the MATH dataset. Left: Accuracy variation as a function of temperature for three baselines and CAP-CoT under different optimization rounds. Right: Mean variation of each method aggregated over the tested temperature range, with the dashed line marking the variation threshold of 1.0.

Limitations and Future Work

While highly effective, the paper notes a few limitations:

  • Prompt Growth: The iterative addition of rules causes the prompt length to grow, though this is capped by a hyperparameter ($K$) to prevent context bloat.
  • Fixed Optimization Horizon: The main experiments rely on a fixed budget of 3 rounds. Implementing a dynamic stopping criterion based on prompt convergence could optimize token usage during the training phase.
  • Application to Specialized Reasoning Models: The framework was validated on standard LLMs and has not yet been extensively tested on newer, specialized long-form reasoning models that utilize internal, hidden thinking traces.

Summary updated

Abstract

Chain-of-Thought (CoT) prompting has emerged as a simple and effective way to elicit step-by-step solutions from large language models (LLMs). However, CoT reasoning can be unstable across runs on long, multi-step problems, leading to inconsistent answers for unchanged task. Most prior work focuses on improving the forward reasoning chain within a single pass, with less attention to iterative and contrastive correction. To address this gap, we propose CAP-CoT, a Cycle Adversarial Prompt optimization framework designed to improve both CoT reasoning accuracy and stability of a single deployed solver. In each cycle, a forward solver generates candidate reasoning chains, an adversarial challenger constructs plausible but deliberately flawed chains using targeted error strategies, and a feedback agent contrasts the two chains and produces step-aligned structured feedback. This feedback closes the optimization loop in two directions, including updating the solver prompt based on errors exposed by the challenger, and updating the challenger prompt to generate increasingly targeted errors in subsequent cycles. Unlike safety-oriented adversarial prompting such as jailbreak or prompt-injection attacks, our adversarial component is task-semantic and aims to expose logical vulnerabilities in reasoning chains. Experiments across six benchmarks and four LLM backbones demonstrate that within two to three adversarial prompt optimization cycles, CAP-CoT consistently reduces variability across runs while improving reasoning accuracy and robustness to prompt perturbations.
BibTeX
@misc{chen2026cap,
  author = {Shuxu Chen and Yitian Zhou and Jiaquan Zhang and Haoyu Bian and Wenrui Hu and Aming Wu and Sungyoung Lee and Chaoning Zhang and Hyundong Shin},
  title = {{CAP-CoT: Cycle Adversarial Prompt for Improving Chain of Thoughts in LLM Reasoning}},
  year = {2026},
  eprint = {2604.23270},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2604.23270},
}

Loading related content...

Figure