Directional Reasoning Injection for Fine-Tuning MLLMs

Chao Huang, Zeliang Zhang, Jiang Liu, Ximeng Sun, Jialian Wu, Xiaodong Yu, Ze Wang, Chenliang Xu, Emad Barsoum, Zicheng Liu

Abstract

Multimodal Large Language Models (MLLMs) often excel at visual perception but lag behind text-only counterparts in complex reasoning tasks. While supervised fine-tuning (SFT) and reinforcement learning (RL) can bridge this gap, they are computationally expensive. Model merging offers a cheaper alternative but frequently destabilizes visual alignment. We introduce DRIFT (Directional Reasoning Injection for Fine-Tuning), a method that injects reasoning capabilities into the gradient space during a brief fine-tuning phase. DRIFT achieves state-of-the-art reasoning performance in just two hours using only 4,000 training examples.

Introduction

Modern MLLMs are proficient at identifying objects and describing scenes, yet they often falter when required to perform multi-step logical or mathematical reasoning based on visual input. To address this, researchers have explored transferring the "thinking" capabilities of reasoning-optimized text models (like the DeepSeek-R1 series) into multimodal architectures. However, traditional methods like massive SFT or RL require significant data and compute. Simple weight merging, while efficient, often fails to maintain the delicate balance between perception and logic, particularly in advanced model families like Qwen.

The Challenge: Why Merging Fails

Model merging typically involves interpolating the weights of a reasoning expert with a multimodal model. While this works reasonably well for LLaVA-style models, it often degrades performance in Qwen-based models. The authors identify a significant "parameter discrepancy" as the root cause: the weights of the reasoning expert and the multimodal base have drifted too far apart for simple averaging to be effective.

Figure 5: Analysis of parameter discrepancies between different model families, showing high L2 norm differences and low cosine similarity in Qwen-based models compared to LLaVA.

As shown in the analysis above, the discrepancy is particularly pronounced in the MLP and Attention layers. This architectural drift means that hard weight merges act as a "replacement" rather than an "enhancement," often breaking the model's existing visual perception capabilities.

Methodology: DRIFT

To overcome the limitations of parameter-space merging, DRIFT operates in the gradient space. The process involves three key steps:

  1. Computing the Reasoning Prior ($\Delta$): A "reasoning vector" is calculated as the difference between a text-only reasoning expert and its multimodal counterpart.
  2. Directional Injection: During a short multimodal fine-tuning session, the standard gradients ($g$) are modified by adding the reasoning prior: $\tilde{g} = g + \alpha \cdot \text{scale}(g, \Delta)$. This "nudges" the model toward reasoning-heavy configurations without the instability of a hard merge.
  3. Selective Application: The injection is targeted at specific transformer components, primarily the attention projections, which are most sensitive to reasoning signals.

Experimental Results

DRIFT was evaluated on rigorous benchmarks including MathVista, MathVision, and MathVerse. The results demonstrate a significant leap in both efficiency and performance.

Figure 1: Comparison of DRIFT against other methods, highlighting its superior accuracy despite using significantly less training data and time.

Key Findings:

  • Efficiency: DRIFT reaches competitive performance in approximately 2 hours on a single node, whereas specialized reasoning models often require days of training on massive clusters.
  • Data Parsimony: The method requires only 4K training examples—a fraction of the 59K to 270K examples used by competing SFT-heavy models.
  • Performance Gains: On MathVista, DRIFT improved the Qwen2.5-VL-7B baseline by +2.4 points, while traditional merging techniques actually caused performance to drop below the baseline.
  • Benchmark Superiority: On the MathVerse "Vision-Only" subset, DRIFT achieved a score of 43.7, outperforming specialized models like R1-OneVision (40.0).

Conclusion

DRIFT represents a shift in how we approach model enhancement, moving from rigid parameter merging to flexible gradient-space guidance. By treating reasoning as a directional "prior" during fine-tuning, DRIFT enables the rapid development of high-performing reasoning MLLMs with minimal data and compute. Future work will explore scaling this method to larger model architectures and broader domains such as code generation.

Summary updated

Abstract

Multimodal large language models (MLLMs) have made rapid progress, yet their reasoning ability often lags behind strong text-only LLMs. Bridging this gap typically requires large-scale multimodal reasoning data or reinforcement learning, incurring substantial cost. An appealing alternative is parameter-space model merging between reasoning-enhanced LLMs and MLLMs, but we show that naive merging is fragile: its effectiveness varies widely across model families and can significantly degrade performance (e.g., for Qwen-based MLLMs). We propose Directional Reasoning Injection for Fine-Tuning (DRIFT), a lightweight method that transfers reasoning knowledge in the gradient space while preserving multimodal alignment. DRIFT precomputes a reasoning prior from the parameter differences between text-only reasoning experts and multimodal models, and uses it to bias gradients during supervised fine-tuning. This design retains the simplicity of standard SFT pipelines while enabling efficient and stable reasoning transfer. Experiments on multimodal reasoning benchmarks, including MathVista and MathVerse, show that DRIFT consistently outperforms naive merging and standard SFT, and matches or surpasses training-intensive methods with substantially lower data and compute.
BibTeX
@inproceedings{huang2026directional,
  author = {Chao Huang and Zeliang Zhang and Jiang Liu and Ximeng Sun and Jialian Wu and Xiaodong Yu and Ze Wang and Chenliang Xu and Emad Barsoum and Zicheng Liu},
  title = {{Directional Reasoning Injection for Fine-Tuning MLLMs}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.1015},
}
Figure