Abstract
Post-training Multimodal Large Language Models (MLLMs) for complex reasoning often relies on Reinforcement Learning with Verifiable Rewards (RLVR). However, standard methods like Group Relative Policy Optimization (GRPO) frequently fail to utilize incorrect rollouts effectively. We present CARE (Contrastive Anchored-REflection), a framework that transforms reasoning failures into powerful learning signals. By anchoring updates to the most efficient correct rollout and employing a "reflection" step to repair near-misses, CARE significantly improves performance on benchmarks like MathVista and MMMU-Pro without increasing inference-time overhead.
Introduction
Current RLVR approaches for math and coding tasks often suffer from two primary issues: gradient stall, where the model learns nothing if all rollouts are incorrect, and credit misassignment, where the model may reinforce "spurious chains" that reach the right answer through flawed logic. CARE addresses these by arguing that "near-misses"—incorrect attempts that are semantically close to the truth—are the most informative data points for teaching precise multimodal reasoning.
The CARE Framework
CARE introduces a failure-centric approach to reinforcement learning through several novel components:
Anchored-Contrastive Subgroup
Rather than treating all rollouts equally, CARE identifies an "anchor"—the verified-correct rollout with the shortest rationale. It then constructs a subgroup of "hard negatives" (incorrect rollouts semantically closest to the anchor). This forces the model to contrast correct logic against its most plausible errors.
Reflection-Guided Resampling (RGR)
RGR acts as a training-only "rescue" mechanism. If a group contains a success, the model takes a representative failure, applies a repair cue (e.g., "Identify the failing operation, correct it..."), and attempts to resample. If the "reflected" version succeeds, it replaces the failure in the training batch, converting a near-miss into a high-quality positive example.
All-Negative Rescue and Token Credit
To prevent training freezes when all rollouts are incorrect, CARE uses a "pseudo-anchor" to maintain a contrastive signal. Additionally, a region-weighted token credit scheme prioritizes "thinking" tokens in rationales, reducing signal dilution over long reasoning chains.
Experimental Results
The framework was evaluated on a 49.3K mixture of multimodal prompts (ChartQA, Geometry3K, ViRL39K) across six major benchmarks, including MathVista and MMMU-Pro.
Performance Gains
CARE consistently outperformed established baselines. On the Qwen2.5-VL-7B backbone, CARE improved macro-averaged accuracy by 4.62 points over GRPO. Using the Qwen3-VL-8B-Instruct backbone, the model achieved state-of-the-art results for its size class, reaching 82.1% on MathVista and 46.7% on MMMU-Pro.
Efficiency and Insights
A key advantage of CARE is its inference-time efficiency; because reflection and repair only occur during training, the final model remains a single-decode model. Furthermore, the use of the shortest correct rationale as an anchor naturally encourages the model to be more concise and efficient, reducing the likelihood of hallucinated reasoning steps.
Conclusion and Future Work
CARE demonstrates that failures are feature-rich learning opportunities. By focusing on hard negatives and structured repair, MLLMs can achieve significant reasoning improvements. Future work includes extending CARE to open-ended tasks where rewards are harder to verify and exploring step-level verification to further refine the training signal.
Summary updated