Introduction
Modern AI systems often tackle complex visual tasks using a decoupled architecture: a vision model generates a text description (caption), which a text-only Large Language Model (LLM) then uses to solve the problem. While this approach is modular and interpretable, it frequently suffers from an interface mismatch. Vision models, typically trained on human-centric captions, often omit the granular, quantitative details—such as exact coordinates or specific angle measurements—that a mathematical reasoner requires. Manually annotating "perfect" captions for every possible task is an unscalable challenge.
Adaptive-Clarification Reinforcement Learning (AC-RL)
To bridge this gap, this paper introduces Adaptive-Clarification Reinforcement Learning (AC-RL). This framework treats the interaction between the vision model (the "captioner") and the reasoner as a reinforcement learning problem, where the reasoner's needs provide implicit supervision.
The Training Scaffold and Tiered Rewards
During training, the system is allowed a multi-turn dialogue. If the reasoner finds the initial caption insufficient, it can issue a clarification request (e.g., "What is the value of angle BCA?"). A frozen reference model provides the missing information, allowing the reasoner to attempt the problem again. To encourage the captioner to be comprehensive from the start, the authors implement a tiered reward structure:
- $R = 1.0$: Correct answer achieved in a single pass (no clarification).
- $R = 0.7$: Correct answer achieved after clarification.
- $R = 0$: Incorrect answer.
By imposing a "clarification tax" (the 0.3 reward penalty), the model is pressured to front-load all necessary information into its initial output. At inference, the clarification mechanism is removed, and the model must rely on the more detailed captioning strategy learned during training.
Experimental Results
The AC-RL framework was evaluated using a Qwen-3B captioner across seven diverse benchmarks, including MathVista, MathVision, and DynaMath. The model was trained on ViRL-39K, a visual instruction dataset focused on mathematical reasoning.
Performance Gains
AC-RL demonstrated significant improvements over standalone VLMs and decoupled baselines. On average, the framework improved accuracy by 4.4 points (from 39.0% to 43.4%). The most substantial gains were observed in vision-centric and quantitatively-intensive domains, such as DynaMath (+10.6 points) and MathVerse (+7.9 points).
Efficiency and Diversity
Beyond raw accuracy, AC-RL proved highly effective at reducing information gaps. Analysis showed that if clarification were allowed at test time, the AC-RL model would have required 39% fewer follow-up questions on the MathVerse dataset compared to standard training methods. Furthermore, the tiered reward system helped maintain higher generation diversity, preventing the model from collapsing into repetitive, low-information captioning styles.
Discussion and Future Work
The core insight of this work is that clarification acts as implicit supervision. Instead of requiring explicit human feedback, the reasoner’s own confusion identifies exactly where the information gap lies. This "densifies" sparse rewards, providing a smoother learning gradient than binary success/failure signals.
However, the authors noted that AC-RL is primarily effective at improving perception (the "what") rather than reasoning steps (the "why"). Future work may explore bidirectional adaptation, where both the vision model and the reasoner co-evolve, or multi-turn dialogues with decaying rewards for each additional hint required.
Summary updated