Introduction
The evolution of Multimodal Large Language Models (MLLMs) has shifted toward creating vision agents capable of complex reasoning through interleaved text and visual tool use. However, existing open-source vision agents often struggle with multi-image scenarios and suffer from "visual attention decay"—a phenomenon where the model's focus on visual tokens diminishes as the reasoning chain lengthens. IMAgent addresses these challenges by employing a pure end-to-end reinforcement learning (RL) approach to train a multi-image vision agent equipped with specialized tools for visual confirmation and reflection.
The IMAgent Framework
IMAgent is built upon the Qwen2.5-VL-7B-Instruct base model and trained using the Generalized Proximal Policy Optimization (GRPO) algorithm. Unlike traditional methods that require expensive Supervised Fine-Tuning (SFT) to initialize tool-use capabilities, IMAgent utilizes a "Zero RL" paradigm.
The core of the agent's reasoning capability lies in its Multimodal Chain-of-Thought (MCoT) framework, which integrates two primary visual tools:
- Visual Confirmation (
zoom_in): Allows the agent to focus on specific regions of interest by generating bounding boxes for high-resolution cropping. - Visual Reflection (
lookback_reuse): Enables the agent to re-insert previously seen images into the current context window, effectively forcing the model to re-examine visual evidence during complex multi-turn reasoning.
Overcoming Visual Attention Decay
A critical contribution of this work is the empirical analysis of attention dynamics in VLMs. The authors observe that during long-form reasoning, attention to image tokens typically flatlines, leading the model to hallucinate based on its own generated text. IMAgent's tools act as an "attention defibrillator," creating spikes in visual attention precisely when the model needs to verify details or synthesize information across multiple images.
Reinforcement Learning Strategy
To stabilize training in a Zero RL setting, the authors introduced two key mechanisms:
- Two-Level Mask Strategy: This includes an Action-Level Mask to ignore environment errors during early exploration and a Trajectory-Level Mask to discard entirely failed reasoning loops, preventing noisy gradients from destabilizing the policy.
- Reward Shaping: A "tool use gain" reward is implemented to incentivize the model to explore tool-based strategies. This prevents the model from converging on a sub-optimal policy that ignores tools to avoid early formatting penalties.
Experimental Results
The effectiveness of IMAgent was evaluated on the newly introduced MIFG-QA (Multi-Image Fine-Grained QA) dataset, which contains 10k complex multi-image pairs. IMAgent achieved a state-of-the-art score of 49.21%, significantly outperforming the base Qwen2.5-VL (41.98%) and specialized models like Pixel Reasoner (44.49%).
Furthermore, IMAgent maintains high performance on single-image benchmarks, scoring 88.48% on V Bench and 72.5%* on HR-Bench. These results demonstrate that the multi-image training and RL strategies do not compromise the model's general vision-language capabilities.
Conclusion
IMAgent represents a significant step forward in open-source vision agents. By bypassing SFT and utilizing targeted visual tools within an RL framework, the model learns emergent strategies for multi-image reasoning while successfully mitigating the problem of visual attention decay. This approach paves the way for more robust and autonomous multimodal agents capable of handling real-world visual complexity.
Summary updated