Plan before Solving: Problem-Aware Strategy Routing for Mathematical Reasoning

Shihao Qi, Jie Ma, Ziang Yin, Lingling Zhang, Jian Zhang, Jun Liu, Feng Tian, Tongliang LiuSubmission to ICLR 2026 · Withdrawn

TL;DR

Different math problems demand fundamentally different reasoning strategies—some need natural language proofs, while others require python scripts or external symbolic solvers. Instead of locking a Large Language Model (LLM) into a single reasoning style or wastefully running all of them, PRISM introduces a lightweight "meta-planner" that inspects a problem before execution to dynamically route it to the single best strategy, a dual-strategy verification pass, or a full exploratory ensemble.

What Problem Does This Paper Address?

Current approaches to LLM mathematical reasoning typically commit to a single fixed paradigm:

  1. Natural Language Reasoning (NLR / Chain-of-Thought): Great for conceptual deduction, but error-prone when handling heavy arithmetic.
  2. Code-Augmented Reasoning (CAR / Program-Aided Language Models): Deterministic and precise in computation, but brittle during problem formulation.
  3. Tool-Integrated Reasoning (TIR): Robust for hard multi-step problems, but suffers from high tool orchestration overhead and latency.
  4. Ensemble-Based Reasoning (EBR / Self-Consistency): Generates multiple expensive trajectories and votes post-hoc.

The authors observe two critical issues:

  • No single strategy fits all problems: A strategy like TIR may excel at advanced algebra but underperform simpler code generation on basic word problems.
  • Efficiency-effectiveness trade-offs are ignored: Uniformly executing heavy tool loops or large ensembles wastes significant compute on problems that could be solved via a single short chain-of-thought.

Methods

PRISM (Planning and Routing through Instance-Specific Modeling) decouples reasoning into two stages: offline strategy planning and online targeted execution.

  1. Curating the MathStrat Preference Dataset: The authors ran all four reasoning strategies across ~13,000 math problems. Each trajectory was scored along three axes:
  2. Correctness ($corr \in \{0, 1\}$)
  3. Process Quality ($qual \in [0, 1]$, penalizing logical fallacies and redundancy)
  4. Efficiency ($eff \in [0, 1]$, computed from normalized token length and latency) These metrics were aggregated into a suitability score and converted into a temperature-scaled soft target distribution over strategies.

  5. Training the Strategy Adapter (SA): A lightweight 1.5B parameter model is trained to predict strategy suitability distributions using a combination of Kullback-Leibler (KL) divergence (matching the soft profile) and cross-entropy loss (anchoring top-1 prediction).

  6. Inference-Time Adaptive Routing Policy: Based on the top predicted probability ($p_{\max}$) and runner-up probability ($p_{\text{2nd}}$), the model selects an execution pathway using two tuned thresholds ($\tau_c = 0.4$, $\tau_a = 0.08$):
  7. Confident Routing ($p_{\max} \ge \tau_c$ and $p_{\max} - p_{\text{2nd}} \ge \tau_a$): High confidence with a clear winner; runs only the single top-ranked strategy.
  8. Deliberative Routing ($p_{\max} \ge \tau_c$ and $p_{\max} - p_{\text{2nd}} < \tau_a$): High confidence but closely matched contenders; executes the top two strategies and verifies consistency.
  9. Exploratory Routing ($p_{\max} < \tau_c$): High uncertainty; runs all strategies and aggregates answers via majority voting.
 Problem ---> [Strategy Adapter] ---> Distribution {p_NLR, p_CAR, p_TIR, p_EBR}
                                             |
             +-------------------------------+------------------------------+
             | (High conf, clear lead)       | (High conf, close scores)    | (Low conf)
             v                               v                              v
      [Confident Mode]              [Deliberative Mode]             [Exploratory Mode]
      Single Strategy Run            Top-2 Verification              All Strategies Vote

Datasets

  • Training: MathStrat, a newly curated dataset of ~13,000 instances built from standard benchmarks (including MATH and GSM8K) annotated with correctness, process quality, and latency profiles across four strategies.
  • Evaluation: Evaluated across five benchmark datasets covering elementary math, competition algebra, and word problems:
  • MATH500 (Olympiad/high-school level)
  • GSM8K (Grade-school arithmetic)
  • AQUA-RAT (Algebraic word problems with rationales)
  • SVAMP & ASDiv (Diverse math word problems)
  • Hungarian Math (used in component ablation studies)

Key Insights and Ideas

  • Pre-execution routing beats post-hoc ensembling: Predicting the best strategy beforehand provides a better accuracy–compute trade-off than naively generating multiple answers across paradigms and aggregating them afterwards.
  • Weaker models gain more from routing: Relative improvements over the single best strategy were +0.9% on Qwen2.5-Math-7B, +2.9% on Deepseek-Math-7B, and +7.6% on Llama-3-8B. Strategic planning mitigates architectural weaknesses.
  • Difficulty dictates discrimination: For simple problems (Levels 1–2), all strategies succeed (confident single-path suffices 90%+ of the time). At moderate difficulty (Levels 3–4), strategy performance diverges sharply (up to a 30% gap between TIR and CAR), making adaptive routing most valuable. At extreme difficulty (Level 5), the system shifts heavily into exploratory multi-path execution.

Key Results

  • Superior Benchmark Accuracy: PRISM achieves state-of-the-art results across all evaluated base models, reaching 70.7% average Pass@1 on Qwen2.5-Math-7B (up from 61.9% on TIR and 22.9% on the Hybrid baseline). On MATH500, it achieves 46.2% Pass@1, outperforming TIR (41.4%) and standard Chain-of-Thought (21.2%).
  • Balanced Compute Overhead: In efficiency ablations, PRISM reached 33.3% accuracy at 5,600 ms inference time, whereas intermediate heuristic setups consumed 6,000–6,300 ms while achieving only 10.0%–17.8% accuracy.
  • Model Scalability: On Qwen2.5 models scaled from 1.5B to 72B parameters, PRISM maintains consistent performance leads on both GSM8K (74.2% $\to$ 95.2%) and MATH500 (32.3% $\to$ 78.4%).

Limitations and Future Work

  • Surface-Feature Heuristics: The Strategy Adapter sometimes relies on surface-level keywords (e.g., detecting trigonometry symbols and defaulting to tool usage), overestimating strategy suitability even when the solver fails downstream during symbolic manipulation.
  • Tie-Breaking Vulnerabilities: In dual-path mode, ties are broken using the planner's prior confidence rather than runtime validation. If the higher-confidence strategy outputs a floating-point formatting artifact or malformed number while the second strategy is correct, PRISM can choose the wrong answer.
  • Future Directions: Incorporating real-time runtime validation signals, tool execution feedback, and richer answer-equivalence normalizers into the decision loop.

Summary updated

Abstract

Existing methods usually leverage a fixed strategy, such as natural language reasoning, code-augmented reasoning, tool-integrated reasoning, or ensemble-based reasoning, to guide Large Language Models (LLMs) to perform mathematical reasoning. Our analysis reveals that the single strategy cannot adapt to problem-specific requirements and thus overlooks the trade-off between effectiveness and efficiency. To address these issues, we propose Planning and Routing through Instance-Specific Modeling (PRISM), a novel framework that decouples mathematical reasoning into two stages: strategy planning and targeted execution. Specifically, we first curate a multi-strategy preference dataset, which we call MathStrat, capturing correctness, process quality, and computational efficiency for each problem–strategy pair. Then, we train a lightweight Strategy Adapter based on the dataset to obtain confidence distributions over the mentioned four reasoning strategies. At inference time, an adaptive routing policy dynamically tailors the reasoning approach based on predictor confidence. It directs the model to use single-strategy execution for high-confidence predictions, dual-strategy verification for competitive scenarios, or comprehensive multi-strategy exploration for uncertain cases. Extensive experiments across five mathematical reasoning benchmarks demonstrate that PRISM consistently outperforms individual strategies and ensemble baselines, achieving improvements ranging from 0.9% to 7.6% across different base models. The adaptive routing approach shows particularly strong benefits for mathematical reasoning tasks across diverse model architectures. Our code is released at https://anonymous.4open.science/r/PRISM-1EAE/.
BibTeX
@misc{qi2025plan,
  author = {Shihao Qi and Jie Ma and ZIANG YIN and Lingling Zhang and Jian Zhang and Jun Liu and Feng Tian and Tongliang Liu},
  title = {{Plan before Solving: Problem-Aware Strategy Routing for Mathematical Reasoning}},
  year = {2025},
  eprint = {2509.24377},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2509.24377},
}
Figure