TL;DR
The paper introduces MSCR (Multi-Source Candidate Replacement), an automated framework that reveals the extreme fragility of Large Language Models (LLMs) in mathematical reasoning. By altering just a single word in a math problem while preserving its meaning, MSCR slashes model accuracy by up to 50% and triggers a "response length collapse," forcing models into highly redundant and computationally expensive reasoning paths.
What Problem Does This Paper Address?
State-of-the-art LLMs achieve near-perfect scores on standard mathematical benchmarks like GSM8K and MATH. However, a lingering question remains: are these models truly executing deep mathematical reasoning, or are they simply master pattern-matchers?
Existing adversarial attack methods used to probe model robustness typically target text classification models, rely heavily on manual prompt engineering, or fail to preserve the original semantics of the question. To bridge this gap, the authors address the lack of automated, scalable, and semantic-preserving robustness evaluations specifically tailored for generative LLMs tackling complex mathematical tasks.
Methods
The authors propose MSCR, an automated adversarial attack method based on multi-source candidate replacement. Instead of relying on humans to rewrite questions, MSCR automatically identifies influential words and generates high-quality substitute candidates by combining three distinct information channels:
- Model Embedding Space: Calculates the cosine similarity of normalized word vectors within the LLM's own vocabulary to find close semantic matches.
- WordNet Dictionary: Leverages a human-curated semantic network to extract clean, lexicographical synonyms.
- Masked Language Model (MLM): Uses an external MLM (
bert-large-uncased) to predict contextually fitting replacement words based on surrounding text.
Candidates from these sources are combined, filtered to remove syntax noise, and substituted into the question one by one. If a substitution causes the target LLM to output an incorrect answer, it is flagged as a preliminary success. To prevent false positives, a secondary advanced commercial model (qwen3-max-preview) acts as a judge to confirm that the perturbed question is still mathematically valid and that the target model genuinely failed.
Datasets
The framework was evaluated using two prominent mathematical reasoning benchmarks:
- GSM8K: A dataset of high-quality, linguistically diverse grade-school math word problems.
- MATH500: A subset of the challenging MATH benchmark, representing advanced, multi-step mathematical problem-solving.
The attacks were systematically tested across 12 open-source LLMs (including members of the Qwen, DeepSeek, Llama, and Gemma families) and transferred to two commercial frontier models (OpenAI-o3 and GPT-4o).
Key Insights and Ideas
- The Illusion of Math Competence: Minor, non-mathematical changes—such as changing the proper noun "Josh" to the pronoun "he", or capitalizing the word "by" to "BY"—completely derail the reasoning chain of top-tier models. This strongly indicates that LLMs remain heavily dependent on surface-level pattern matching rather than abstract logical deduction.
- Response Length Collapse: When confronted with a perturbed question, attacked models don't just fail quickly; their reasoning paths blow up. The generated Chain-of-Thought (CoT) responses often become highly circular and redundant, sometimes stretching to 4x to 10x the original token length.
- The Paradox of Weaker Models: Interestingly, weaker models (like the
gemma-2series) showed minimal change in response length under attack. The authors hypothesize that because these models lack true reasoning capabilities, they rely entirely on memorized templates. When a template match fails, they fail immediately without attempting alternative, convoluted reasoning paths.
Key Results
The quantitative impact of changing just one word is stark across all tested models:
- Massive Accuracy Drops: On the GSM8K benchmark,
Qwen2.5-Math-1.5B-Instructsuffered the most severe degradation, with its accuracy plummeting by 49.89%. On MATH500,gemma-2-9b-itsaw its accuracy slashed by 35.40%. - Strong Cross-Model Transferability: Adversarial examples generated to fool an open-source model (
gemma-3-27b-it) successfully compromised commercial frontier models. When evaluated on these perturbed samples, OpenAI-o3 and GPT-4o experienced accuracy drops of 15.88% and 17.21% on GSM8K, respectively.
Limitations and Future Work
While the paper comprehensively demonstrates how easily LLMs can be misled, it primarily focuses on exposing these vulnerabilities rather than fixing them. The authors do not propose a concrete defense mechanism (such as adversarial fine-tuning or robust decoding strategies) within this work. They note that the primary value of MSCR is providing an automated evaluation framework to help future researchers understand these underlying reasoning flaws and engineer more resilient, efficient defense strategies.
Summary updated