CARE What Fails: Contrastive Anchored-REflection for Verifiable Multimodal

Yongxin Wang, Zhicheng Yang, Meng Cao, Mingfei Han, Haokun Lin, Yingying Zhu, Xiaojun Chang, Xiaodan LiangarXiv 2025 · DOI · arXiv

Abstract

Post-training Multimodal Large Language Models (MLLMs) for complex reasoning often relies on Reinforcement Learning with Verifiable Rewards (RLVR). However, standard methods like Group Relative Policy Optimization (GRPO) frequently fail to utilize incorrect rollouts effectively. We present CARE (Contrastive Anchored-REflection), a framework that transforms reasoning failures into powerful learning signals. By anchoring updates to the most efficient correct rollout and employing a "reflection" step to repair near-misses, CARE significantly improves performance on benchmarks like MathVista and MMMU-Pro without increasing inference-time overhead.

Introduction

Current RLVR approaches for math and coding tasks often suffer from two primary issues: gradient stall, where the model learns nothing if all rollouts are incorrect, and credit misassignment, where the model may reinforce "spurious chains" that reach the right answer through flawed logic. CARE addresses these by arguing that "near-misses"—incorrect attempts that are semantically close to the truth—are the most informative data points for teaching precise multimodal reasoning.

The CARE Framework

CARE introduces a failure-centric approach to reinforcement learning through several novel components:

Overview of the CARE framework, highlighting the Anchored-Contrastive selection and Reflection-Guided Resampling components.

Anchored-Contrastive Subgroup

Rather than treating all rollouts equally, CARE identifies an "anchor"—the verified-correct rollout with the shortest rationale. It then constructs a subgroup of "hard negatives" (incorrect rollouts semantically closest to the anchor). This forces the model to contrast correct logic against its most plausible errors.

Reflection-Guided Resampling (RGR)

RGR acts as a training-only "rescue" mechanism. If a group contains a success, the model takes a representative failure, applies a repair cue (e.g., "Identify the failing operation, correct it..."), and attempts to resample. If the "reflected" version succeeds, it replaces the failure in the training batch, converting a near-miss into a high-quality positive example.

All-Negative Rescue and Token Credit

To prevent training freezes when all rollouts are incorrect, CARE uses a "pseudo-anchor" to maintain a contrastive signal. Additionally, a region-weighted token credit scheme prioritizes "thinking" tokens in rationales, reducing signal dilution over long reasoning chains.

Experimental Results

The framework was evaluated on a 49.3K mixture of multimodal prompts (ChartQA, Geometry3K, ViRL39K) across six major benchmarks, including MathVista and MMMU-Pro.

Overall reward comparison over training steps for CARE, CARE without RGR, and the GRPO baseline.

Performance Gains

CARE consistently outperformed established baselines. On the Qwen2.5-VL-7B backbone, CARE improved macro-averaged accuracy by 4.62 points over GRPO. Using the Qwen3-VL-8B-Instruct backbone, the model achieved state-of-the-art results for its size class, reaching 82.1% on MathVista and 46.7% on MMMU-Pro.

Efficiency and Insights

A key advantage of CARE is its inference-time efficiency; because reflection and repair only occur during training, the final model remains a single-decode model. Furthermore, the use of the shortest correct rationale as an anchor naturally encourages the model to be more concise and efficient, reducing the likelihood of hallucinated reasoning steps.

Conclusion and Future Work

CARE demonstrates that failures are feature-rich learning opportunities. By focusing on hard negatives and structured repair, MLLMs can achieve significant reasoning improvements. Future work includes extending CARE to open-ended tasks where rewards are harder to verify and exploring step-level verification to further refine the training signal.

Summary updated

Abstract

Group-relative reinforcement learning with verifiable rewards (RLVR) often wastes the most informative data it already has the failures. When all rollouts are wrong, gradients stall; when one happens to be correct, the update usually ignores why the others are close-but-wrong, and credit can be misassigned to spurious chains. We present CARE (Contrastive Anchored REflection), a failure-centric post-training framework for multimodal reasoning that turns errors into supervision. CARE combines: (i) an anchored-contrastive objective that forms a compact subgroup around the best rollout and a set of semantically proximate hard negatives, performs within-subgroup z-score normalization with negative-only scaling, and includes an all-negative rescue to prevent zero-signal batches; and (ii) Reflection-Guided Resampling (RGR), a one-shot structured self-repair that rewrites a representative failure and re-scores it with the same verifier, converting near-misses into usable positives without any test-time reflection. CARE improves accuracy and training smoothness while explicitly increasing the share of learning signal that comes from failures. On Qwen2.5-VL-7B, CARE lifts macro-averaged accuracy by 4.6 points over GRPO across six verifiable visual-reasoning benchmarks; with Qwen3-VL-8B it reaches competitive or state-of-the-art results on MathVista and MMMU-Pro under an identical evaluation protocol.
BibTeX
@misc{wang2025care,
  author = {Yongxin Wang and Zhicheng Yang and Meng Cao and Mingfei Han and Haokun Lin and Yingying Zhu and Xiaojun Chang and Xiaodan Liang},
  title = {{CARE What Fails: Contrastive Anchored-REflection for Verifiable Multimodal}},
  year = {2025},
  eprint = {2512.19554},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2512.19554},
}
Figure