Beyond Correctness: Benchmarking and Aligning Response Behaviors in Hybrid-Thinking MLLMs

Xinming Wang, Weinong Wang, Hongming Yang, Yansong Lin, Zheng Ruan, Shangpin Peng, Qiming Peng, Nan Qiao, Fengyuan Lu, Guoqing Ma, Marito Li, Songyang Zhang, Saiyong Yang, Han Hu, Yonglong Tian, Xu-Yao ZhangarXiv 2026 · arXiv

TL;DR

Modern multimodal large language models (MLLMs) increasingly use "hybrid-thinking" architectures, allowing them to toggle between slow, deliberative reasoning and fast, direct answers. This paper reveals a critical catch: fast "non-thinking" modes frequently suffer from severe response-pattern failures, such as leaking internal monologues, repeating text, and introducing logical contradictions. To address this, the authors introduce PatternEval, a benchmark to diagnose these behavioral flaws, and PatternRL, a reinforcement learning framework that successfully trains models to deliver clean, concise answers without sacrificing accuracy.

What Problem Does This Paper Address?

State-of-the-art MLLMs are increasingly deployed with hybrid interfaces to balance reasoning capability with computational efficiency. When a user presents a complex problem, the model allocates extra test-time tokens to "think" through it step-by-step. Conversely, for latency-sensitive interactions, it switches to a "non-thinking" mode to output a direct answer.

However, the authors find that optimizing post-training purely for correctness causes a severe response-pattern misalignment between these two interfaces. Even when a model arrives at the correct final answer in its fast non-thinking mode, its output behavior often degrades—exposing internal chain-of-thought (CoT) text, repeating sentences, or generating empty, unsupported reasoning wrappers. This paper aims to define, benchmark, and mitigate these user-visible behavioral failures.

Examples of the four recurrent response-pattern failures evaluated in PatternEval.

Methods

To bridge the gap between thinking and non-thinking interfaces, the authors propose a post-training alignment pipeline consisting of two main components:

  1. PatternRM (Reward Model): The authors built a text-only reward model initialized from Qwen3.5-27B. It is trained via supervised fine-tuning (SFT) on a curated dataset of 90K model responses annotated by a consensus of frontier models. PatternRM learns to explicitly detect the four core failure modes (CoT leakage, repetition, contradiction, and performative reasoning) using only text inputs, making it highly computationally efficient during RL training.
  2. PatternRL (Reinforcement Learning): Using Group Relative Policy Optimization (GRPO), the authors integrate pattern-specific penalties from PatternRM directly into the RL reward function alongside standard correctness verifiers. The reward structure ensures that incorrect answers always receive zero reward, while correct answers that exhibit poor behavioral patterns face a fractional penalty, steering the model toward concise and clean outputs.

Datasets

The paper introduces PatternEval, a failure-enriched diagnostic benchmark consisting of 2,415 multimodal prompts designed to stress-test response-pattern robustness. The benchmark spans nine constituent categories organized into three broad task families:

  • Visual Perception and Grounding (VG): 1,181 prompts covering out-of-distribution perception, content recognition, grounding, and factuality.
  • OCR and Structured Image Understanding (OS): 579 prompts covering text-centric OCR and chart understanding.
  • Multimodal Knowledge Reasoning (KR): 655 prompts spanning STEM, knowledge, and multi-step reasoning.
The composition of PatternEval and the pattern misalignment observed across flagship models.

Key Insights and Ideas

  • Capability Does Not Guarantee Alignment: Simply scaling up models or achieving high task accuracy does not automatically fix response-pattern failures. Even flagship frontier models exhibit pronounced thinking–non-thinking misalignment, with non-thinking failure trigger rate gaps reaching up to 48.64%.
  • The Length-Failure Correlation: Response-pattern failures are strongly correlated with response length. Longer answers generated in non-thinking mode are significantly more prone to breaking formatting rules, leaking internal monologues, or introducing logical contradictions.
  • Asymmetric Failure Distribution: Chain-of-thought (CoT) leakage and response repetition are the most dominant and frequent failure modes across all tested models, whereas hard logical contradictions occur less frequently but require much finer-grained semantic assessment to detect.

Key Results

  • Widespread Misalignment: Evaluation across 25 distinct model configurations showed that non-thinking modes consistently trigger higher failure rates than their thinking counterparts across all task families.
  • PatternRL Effectiveness: When evaluated on Qwen3-VL-4B and Qwen3-VL-8B, PatternRL successfully reduced non-thinking failure trigger rates by 13.08 and 14.35 percentage points, respectively, compared to a correctness-only baseline (BaseRL).
  • Minimal Accuracy Trade-off: Crucially, this behavioral alignment came at a marginal cost to task performance, with aggregate accuracy across general reasoning tasks shifting by less than one percentage point.

Limitations and Future Work

  • RL-Stage Limitations: While PatternRL significantly mitigates response-pattern failures, it does not eliminate them entirely. The authors note that a residual gap remains, suggesting that these bad habits are deeply embedded during earlier pre-training or SFT stages.
  • Capacity-Dependent Trade-offs: Smaller models (like the 4B variant) experience a more pronounced accuracy drop when constrained by PatternRL. This happens because verbose or exploratory "thinking-like" behaviors might actually help smaller models stumble onto the correct answer; penalizing these behaviors restricts their effective rollout space.

The authors suggest that future work should focus on injecting response-pattern constraints earlier in the training pipeline (during mid-training or SFT) rather than relying solely on lightweight auxiliary rewards during reinforcement learning.

Summary updated

Abstract

Hybrid-thinking multimodal large language models (MLLMs) allow a single model to alternate between deliberative thinking and latency-efficient non-thinking inference. Although these modes differ in reasoning budget, their delivered responses should satisfy the same user-facing standard. Correctness alone may not characterize this response quality; we therefore evaluate task accuracy and response-pattern failures as complementary outcomes. We study this gap through \textbf{response-pattern alignment}: whether thinking and non-thinking interfaces preserve acceptable final-response behavior. We introduce \textbf{PatternEval}, a failure-enriched diagnostic benchmark comprising 2,415 multimodal prompts spanning visual perception and grounding, structured image understanding, and multimodal knowledge reasoning. PatternEval tests four recurrent failures: chain-of-thought leakage, response repetition, logical contradiction, and performative reasoning. Response-pattern failures are widespread across models from different providers, with non-thinking inference exhibiting substantially higher failure rates and thereby creating systematic misalignment between thinking and non-thinking interfaces. Motivated by this diagnosis, we develop \textbf{PatternRM}, a response-level reward model, and \textbf{PatternRL}, which introduces pattern-specific penalties during reinforcement learning. Experiments on Qwen3-VL-4B and Qwen3-VL-8B show that incorporating pattern-specific penalties into reinforcement learning can mitigate cross-mode misalignment while incurring a marginal task performance trade-off. Together, PatternEval and PatternRL provide an evaluation-and-training framework for aligning user-visible response patterns across hybrid-thinking interfaces.
BibTeX
@misc{wang2026beyond,
  author = {Xinming Wang and Weinong Wang and Hongming Yang and Yansong Lin and Zheng Ruan and Shangpin Peng and Qiming Peng and Nan Qiao and Fengyuan Lu and Guoqing Ma and Marito Li and Songyang Zhang and Saiyong Yang and Han Hu and Yonglong Tian and Xu-Yao Zhang},
  title = {{Beyond Correctness: Benchmarking and Aligning Response Behaviors in Hybrid-Thinking MLLMs}},
  year = {2026},
  eprint = {2608.12781},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2608.12781},
}
Figure