Reasoning-Intensive Regression

Diane Tchuindjo, Omar KhattabSubmission to ICLR 2026 · Rejected

TL;DR

Standard language regression methods struggle when predicting continuous scores that require multi-step deduction, as small fine-tuned encoders often collapse to mean predictions while prompted LLMs suffer from coarse output quantization. To bridge this gap, the authors introduce MENTAT, a lightweight framework that couples batch-reflective prompt optimization with a tiny neural aggregator over multiple rollouts, boosting performance by up to 65% over traditional baselines.

What Problem Does This Paper Address?

In natural language processing, regression tasks traditionally focus on surface-level or moderate semantic understanding—such as sentiment analysis, text similarity, or document ranking. However, emerging applications increasingly demand Reasoning-Intensive Regression (RiR): deriving precise, continuous numerical scores from text where each sample requires deep, multi-step logical deduction or complex evaluative judgment. Examples include pinpointing the exact step where a mathematical proof breaks down or judging fine-grained rubric compliance in long-form generation.

Practitioners face two main bottlenecks in RiR:

  1. The Small-Data / Low-Compute Constraint: Tasks are often ad-hoc and lack massive supervised datasets or the compute budget required for full reinforcement learning or massive fine-tuning.
  2. The Failure Modes of Standard Approaches: Small encoder models (e.g., NeoBERT) often "hack" distance-based losses by collapsing to mean predictions. Conversely, frozen LLMs can reason through steps, but because they generate text, their numerical outputs are heavily quantized into coarse, human-preferred numbers (like .00 or .50), ruining continuous precision.

Methods

The authors propose MENTAT (Mistake-Aware prompt Evolver with Neural Training And Testing), a two-phase framework:

  1. Phase 1: Iterative Prompt Evolution
    Starting from an intentionally minimal prompt, the LLM evaluates a batch of training instances. In each of 3 sequential iterations, the LLM analyzes the $\sqrt{n}$ worst-performing examples (based on absolute error) alongside an explicit log of prior optimization attempts. It then generates refined instructions designed to address recurring failure modes and provide calibration guidelines. The best-performing prompt is chosen on a validation split using the Concordance Correlation Coefficient (CCC).

  2. Phase 2: Multi-Rollout Generation & Neural Aggregation
    Using the evolved prompt, the LLM draws $K=3$ independent rollouts per input to capture reasoning diversity and uncertainty. Instead of taking a simple average, MENTAT passes the sorted predictions and summary statistics (mean, standard deviation, min, max) to a lightweight Multi-Layer Perceptron (MLP) with a single 8-unit hidden layer. The MLP is trained on combined CCC and Normalized Mean Square Error (NMSE) loss to map discrete, stochastic LLM outputs onto calibrated, continuous scores.

Datasets

The authors construct an initial RiR benchmark across four distinct domains:

  • Mathematical Error Detection: Adapted from ProcessBench, this task asks models to identify the first erroneous step in an incorrect math solution and predict the continuous fraction ($0.0$ to $10.0$) of the solution preceding the error.
  • Instruction Following: Derived from the WILDIFEVAL corpus, models predict the overall adherence ($0.0$ to $1.0$) of a Llama-3.1-8B response across multiple fine-grained instructions (scored via the harmonic mean of atomic checks).
  • Pairwise RAG Comparison: Derived from RAG-QA Arena, requiring models to predict human preference margins (ranging from $-2.0$ to $+2.0$) comparing two RAG outputs on helpfulness, truthfulness, and completeness.
  • Essay Grading: Derived from the ELLIPSE corpus (student essays scored $1.0$ to $5.0$). This serves as a reference task representing moderate semantic understanding rather than heavy sequential deduction.

Key Insights and Ideas

  • NMSE Can Mask Model Collapse: Standard distance metrics like NMSE reward models that play it safe near the mean. A fine-tuned NeoBERT achieved an NMSE of $1.01$ on math error detection simply by collapsing to the dataset average (CCC of $0.01$). Concordance Correlation Coefficient (CCC) is essential because it penalizes systematic bias and rewards preserving natural variance.
  • Severe Output Quantization in LLMs: Directly querying LLMs for continuous numbers results in extreme discretization. On math error detection, $63.1\%$ of GPT-4.1 predictions and $86.5\%$ of GPT-5 predictions ended in .00 or .50, despite ground truth labels being roughly uniform.
  • "Overthinking" in Large Reasoning Models: On simpler evaluative tasks like Pairwise RAG, GPT-5 underperformed GPT-4.1 (CCC of $0.31$ vs. $0.47$). Chain-of-thought prompting caused GPT-5 to over-deliberate, hedge, and cluster predictions near zero rather than utilizing the full margin scale.
  • Complementary Synergy: Prompt evolution stabilizes reasoning patterns (reducing rollout variance by $30\%$ on math errors), while the downstream neural aggregator smooths out quantization into continuous scalar outputs.

Key Results

  • Math Error Detection (500 samples): GPT-5 with MENTAT achieved a 0.78 CCC and 0.42 NMSE, markedly outperforming detailed human prompts ($0.69$ CCC / $0.78$ NMSE) and fine-tuned NeoBERT ($0.06$ CCC / $1.01$ NMSE).
  • Instruction Following: On an open-source model (gpt-oss-20b), MENTAT raised the CCC from $0.32$ (basic prompt) to 0.42–0.43, lowering NMSE from $1.18$ to 0.90–0.95.
  • Pairwise RAG Comparison: GPT-4.1 equipped with MENTAT outperformed both detailed prompting and GPT-5 variants, improving NMSE from $2.20$ to 0.80 and reaching a CCC of 0.52.
  • Wall-Clock Optimization Efficiency: MENTAT’s batched 3-iteration design converged in approximately $8\times$ fewer sequential optimization steps than modern evolutionary optimizers like GEPA.

Limitations and Future Work

  • Inference Overhead: Generating 3 rollouts per prediction triples token consumption at inference time compared to single-pass prompting.
  • Optimization Scope: MENTAT currently targets single-prompt pipelines along a single trajectory; extending it to multi-stage LLM programs or multi-trajectory restarts remains unexplored.
  • Residual Annotation Biases: On Pairwise RAG, MENTAT reduced but could not eliminate length bias (correlating at $\rho = 0.551$ with response length gaps), partly because the underlying human ground truth data already harbored verbosity bias ($\rho = 0.332$).

Summary updated

Abstract

AI researchers and practitioners increasingly apply large language models (LLMs) to what we call reasoning-intensive regression (RiR), i.e., deducing subtle numerical scores from text. Unlike standard language regression tasks, e.g., for sentiment or similarity, RiR often appears instead in ad-hoc problems such as rubric-based scoring, modeling dense rewards in complex environments, or domain-specific retrieval, where much deeper analysis of context is required while only limited task-specific training data and computation are available. We cast four realistic problems as RiR tasks to establish an initial benchmark, and use that to test our hypothesis that prompting frozen LLMs and finetuning Transformer encoders via gradient descent will both often struggle in RiR. We then propose MENTAT, a simple and lightweight method that combines batch-reflective prompt optimization with neural ensemble learning. MENTAT achieves up to 65% improvement over both baselines, though substantial room remains for future advances in RiR.
BibTeX
@inproceedings{tchuindjo2025reasoning,
  author = {Diane Tchuindjo and Omar Khattab},
  title = {{Reasoning-Intensive Regression}},
  booktitle = {ER @ NeurIPS},
  year = {2025},
  url = {https://openreview.net/forum?id=m3DbJMbpXg},
}
Figure