Bridging Modal Isolation in Interleaved Thinking: Supervising Modality Transitions via Stepwise Reinforcement

Tingyu Li, Le Zhou, Siyuan Li, Yujun Wu, Xinglong Xu, Jingxuan Wei, Conghui He, Cheng TanarXiv 2026 · arXiv

An elegant and structured summary of the paper is provided below.

TL;DR

The paper identifies Modal Isolation as a critical failure mode in unified multimodal models during long-chain interleaved text-image reasoning, where text and image generation progressively decouple and fail to inform one another. To resolve this, the authors introduce MoTiF (Modality Transition Fidelity), a two-stage training framework that optimizes transition-level boundaries via Reflective SFT and Flow-GRPO reinforcement learning, achieving massive task accuracy gains without relying on final outcome rewards.

What Problem Does This Paper Address?

Unified multimodal models (UMMs) can natively generate both text and images, enabling an "interleaved thinking" paradigm where a model alternates between textual reasoning and visual world modeling to solve complex spatial or physical tasks. However, in long-chain scenarios, these models suffer from a persistent failure mode termed Modal Isolation.

As illustrated in Figure 1, information loss compounds at the modality boundaries:

  1. Text-to-Image Loss ($\ell_{\text{text}\to\text{image}}$): The generated image diverges from the textual plan (e.g., drawing a path that passes through a wall in a maze navigation task).
  2. Image-to-Text Loss ($\ell_{\text{image}\to\text{text}}$): The subsequent text generation fails to detect errors in its own generated image, blindly validating an incorrect visual state.

Consequently, the two modalities alternate in form but operate in isolation, leading to highly confident but incorrect final answers.

Figure 1: The phenomenon of modal iso- lation in interleaved thinking. In a maze navigation task, the generated image de- picts an inconsistent path. Subsequently, the model observes the erroneous image yet fails to detect the discrepancy, incor- rectly validating the path.

Methods

The authors propose MoTiF, a two-stage training framework designed to provide process-level supervision directly at the modality boundaries:

  • Decomposition of Interleaved Thinking: Each text segment within a reasoning cycle is broken down into three functional atomic operations: observing the previous image ($t^{\text{obs}}$), performing symbolic reasoning ($t^{\text{rea}}$), and issuing a drawing instruction ($t^{\text{ins}}$).
  • Stage 1: Reflective SFT (Optimizing Image-to-Text): To train the model to detect and recover from visual errors, the authors inject controlled corruptions into intermediate images during training. A frontier model generates "reflective text" that either bypasses the error or re-issues a correct drawing instruction, teaching the model self-correction.
  • Stage 2: Flow-GRPO (Optimizing Text-to-Image): Using reinforcement learning via Group Relative Policy Optimization adapted for flow matching models (Flow-GRPO), the visual generation modality is optimized to strictly adhere to the textual drawing instructions using a rubric-based VLM-as-Judge reward ($R_{\text{img}}$).
Modality transition loss and atomic decomposition

Datasets

The framework is evaluated across four distinct visual puzzle benchmarks requiring long-chain interleaved reasoning:

  • Sokoban: A grid-based planning task where the box-pushing logic places the primary cognitive load on the understanding module.
  • Maze: A path-finding navigation task requiring coordination of text planning and visual tracking.
  • Multi-hop Manipulation: A CLEVR-inspired task where objects are sequentially added, removed, or modified using Blender-rendered scenes.
  • Ball Tracking: A physics-intuition task predicting the final destination of a ball after multiple specular reflections.

The authors synthesized high-quality interleaved fine-tuning datasets for these tasks using Gemini 3.1 Flash-Lite, ranging from 7,300 to 8,000 samples per task domain.

Key Insights and Ideas

  • End-Task Rewards are Misaligned: Optimizing solely for final task accuracy can inadvertently reward models that arrive at correct answers through hallucinated intermediate reasoning, leaving modal isolation unaddressed.
  • The Compounding Error Loop: Text-to-image hallucination and visual utilization deficits mutually amplify each other over long chains, creating a fundamental bottleneck for scalability in multimodal reasoning.
  • Synergistic Evolution: Improving image generation fidelity (Text-to-Image) indirectly benefits the subsequent text observation step. When generated images are more faithful, the model's text-based "observation" step encounters less distributional shift, mitigating text-modality degradation.

Key Results

MoTiF was applied to the Bagel-7B-MoT base model and demonstrated substantial performance gains across all benchmarks:

  • Overall Accuracy: The proposed method achieved an overall absolute accuracy improvement of +45.61% compared to the baseline model.
  • Task-Specific Gains: The maximum accuracy improvements reached +33.33% on Sokoban, +60.44% on Maze, +45.34% on Multi-hop Manipulation, and +43.34% on Ball Tracking.
  • Mitigation of Forgetting: Sequential optimization (Reflective SFT followed by Flow-GRPO) proved highly effective, showing that the gains in image generation fidelity far outweighed any minor degradation in text understanding capabilities.

Limitations and Future Work

  • Dependence on VLM Judges: The transition-level rewards rely on prompting strong frontier VLMs (like Qwen3.5-27B) as judges, which introduces potential evaluation biases or API dependencies.
  • Sequential vs. Joint Optimization: The current framework uses a two-stage sequential pipeline; future work could explore joint multi-task training with careful loss balancing to prevent any potential modality degradation.
  • Generalization: While tested on four structured visual puzzle domains, scaling MoTiF to open-ended, real-world multimodal reasoning tasks remains an open challenge.

Summary updated

Abstract

Interleaved thinking, where a unified multimodal model alternates between textual reasoning and visual generation, has shown promise on spatial and physical tasks. However, in complex long-chain scenarios, we identify a fundamental failure mode: generated images diverge from the textual context while subsequent text ignores the visual evidence, causing the two modalities to alternate without genuinely informing each other. We term this Modal Isolation and attribute it to compounding information loss at modality boundaries. We decompose each reasoning cycle into atomic operations and define modality transition loss, quantifying cross-modal hallucination (text-to-image) and visual utilization deficit (image-to-text) at each boundary. We propose MoTiF (Modality Tiransition Fidelity), a two-stage training framework that directly optimizes these transitions: Reflective SFT trains the model to detect and recover from erroneous visual outputs; Flow-GRPO improves image generation fidelity via reinforcement learning. All training signals in MoTiF derive from transition-level fidelity rather than end-task accuracy. Across four visual puzzle benchmarks, this transition-level supervision substantially improves both cross-modal coherence and final task accuracy. The results demonstrate that effective interleaved reasoning requires explicit structural supervision at modality boundaries, not merely scaling or end-task optimization.
BibTeX
@misc{li2026bridging,
  author = {Tingyu Li and Le Zhou and Siyuan Li and Yujun Wu and Xinglong Xu and Jingxuan Wei and Conghui He and Cheng Tan},
  title = {{Bridging Modal Isolation in Interleaved Thinking: Supervising Modality Transitions via Stepwise Reinforcement}},
  year = {2026},
  eprint = {2606.12886},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2606.12886},
}

Loading related content...

Figure