Abstract
Multimodal Large Language Models (MLLMs) often excel at visual perception but lag behind text-only counterparts in complex reasoning tasks. While supervised fine-tuning (SFT) and reinforcement learning (RL) can bridge this gap, they are computationally expensive. Model merging offers a cheaper alternative but frequently destabilizes visual alignment. We introduce DRIFT (Directional Reasoning Injection for Fine-Tuning), a method that injects reasoning capabilities into the gradient space during a brief fine-tuning phase. DRIFT achieves state-of-the-art reasoning performance in just two hours using only 4,000 training examples.
Introduction
Modern MLLMs are proficient at identifying objects and describing scenes, yet they often falter when required to perform multi-step logical or mathematical reasoning based on visual input. To address this, researchers have explored transferring the "thinking" capabilities of reasoning-optimized text models (like the DeepSeek-R1 series) into multimodal architectures. However, traditional methods like massive SFT or RL require significant data and compute. Simple weight merging, while efficient, often fails to maintain the delicate balance between perception and logic, particularly in advanced model families like Qwen.
The Challenge: Why Merging Fails
Model merging typically involves interpolating the weights of a reasoning expert with a multimodal model. While this works reasonably well for LLaVA-style models, it often degrades performance in Qwen-based models. The authors identify a significant "parameter discrepancy" as the root cause: the weights of the reasoning expert and the multimodal base have drifted too far apart for simple averaging to be effective.
As shown in the analysis above, the discrepancy is particularly pronounced in the MLP and Attention layers. This architectural drift means that hard weight merges act as a "replacement" rather than an "enhancement," often breaking the model's existing visual perception capabilities.
Methodology: DRIFT
To overcome the limitations of parameter-space merging, DRIFT operates in the gradient space. The process involves three key steps:
- Computing the Reasoning Prior ($\Delta$): A "reasoning vector" is calculated as the difference between a text-only reasoning expert and its multimodal counterpart.
- Directional Injection: During a short multimodal fine-tuning session, the standard gradients ($g$) are modified by adding the reasoning prior: $\tilde{g} = g + \alpha \cdot \text{scale}(g, \Delta)$. This "nudges" the model toward reasoning-heavy configurations without the instability of a hard merge.
- Selective Application: The injection is targeted at specific transformer components, primarily the attention projections, which are most sensitive to reasoning signals.
Experimental Results
DRIFT was evaluated on rigorous benchmarks including MathVista, MathVision, and MathVerse. The results demonstrate a significant leap in both efficiency and performance.
Key Findings:
- Efficiency: DRIFT reaches competitive performance in approximately 2 hours on a single node, whereas specialized reasoning models often require days of training on massive clusters.
- Data Parsimony: The method requires only 4K training examples—a fraction of the 59K to 270K examples used by competing SFT-heavy models.
- Performance Gains: On MathVista, DRIFT improved the Qwen2.5-VL-7B baseline by +2.4 points, while traditional merging techniques actually caused performance to drop below the baseline.
- Benchmark Superiority: On the MathVerse "Vision-Only" subset, DRIFT achieved a score of 43.7, outperforming specialized models like R1-OneVision (40.0).
Conclusion
DRIFT represents a shift in how we approach model enhancement, moving from rigid parameter merging to flexible gradient-space guidance. By treating reasoning as a directional "prior" during fine-tuning, DRIFT enables the rapid development of high-performing reasoning MLLMs with minimal data and compute. Future work will explore scaling this method to larger model architectures and broader domains such as code generation.
Summary updated