TL;DR
VITAL introduces a latent-space reasoning framework for medical Multimodal Large Language Models (MLLMs) that utilizes visual-semantic dual supervision during training. By forcing continuous hidden states to encode both textual logic and high-resolution visual evidence, VITAL eliminates the massive inference latency and language bottlenecks of explicit Chain-of-Thought (CoT) reasoning. The framework achieves state-of-the-art performance competitive with trillion-parameter proprietary models while offering post-hoc visual and textual interpretability at zero additional inference cost.
What Problem Does This Paper Address?
Medical Visual Question Answering (VQA) demands highly precise, fine-grained visual understanding. While explicit Chain-of-Thought (CoT) prompting helps models decompose complex diagnoses into structured steps, it suffers from two major flaws in clinical settings:
- The Language Bottleneck: Forcing a model to compress rich visual features (like subtle lesion textures or boundary gradients) into discrete text tokens inevitably discards critical perceptual details.
- Inference Overhead: Generating verbose text reasoning chains significantly increases latency and computational cost, which is highly impractical for time-sensitive clinical environments.
"Latent reasoning"—where a model processes continuous hidden states instead of text tokens—offers a faster alternative. However, existing latent reasoning methods suffer from modality collapse (where hidden states drift into text-only representations), self-referential visual anchors, and a complete lack of interpretability. Clinicians cannot trust a black-box model that provides no explanation for its diagnostic path.
Methods
The authors propose VITAL (Visual-Implicit Thinking with Aligned Latents), which implements a recurrent latent reasoning loop with strict train-inference parity. To prevent modality collapse and ensure clinical transparency, VITAL introduces Visual-Semantic Dual Supervision using temporary training-time "scaffolding":
- Semantic Supervision: An auxiliary text decoder (a lightweight, independent language model) is attached to the latent states to reconstruct human-readable reasoning chains step-by-step.
- Visual Supervision: A visual projector maps the latent states into the feature space of a frozen, independent expert medical vision encoder (MedSigLIP), regressing Region of Interest (ROI) features.
Crucially, both the text decoder and visual projector are entirely discarded at inference time, resulting in zero extra latency, memory, or parameter overhead. However, they can be re-attached post-hoc to act as a "textual window" (showing what the model reasons) and a "visual window" (showing where the model looks), providing dual interpretability.
To stabilize the optimization of these continuous chains, the authors utilize a three-phase curriculum learning strategy that progressively scales the reasoning depth from $K=1$ to $K=4$ steps.
Datasets
The authors constructed a large-scale dataset of ~61K five-tuple training samples (comprising an image, question, answer, $K$-step reasoning chain, and ROI visual feature). The dataset was automatically compiled from two public medical segmentation sources:
- Medical Segmentation Decathlon (MSD)
- BiomedParse
The dataset spans 9 imaging modalities (including CT, MRI, X-ray, ultrasound, and pathology). For evaluation, VITAL was tested across 7 distinct benchmarks: the MSD and BiomedParse test splits, three public medical VQA datasets (VQA-RAD, PathVQA, PMC-VQA), a visual grounding benchmark (GEMeX-RMCoT), and a private, held-out in-house neuroimaging dataset of 1,000 VQA pairs to ensure robust generalization.
Key Insights and Ideas
- Naïve Latent Reasoning Can Degrade Performance: The authors discovered that unconstrained latent reasoning baselines actually perform worse than the zero-shot base model on medical tasks. Without explicit dual supervision, latent states rapidly collapse into text-dominant subspaces, losing all spatial grounding.
- Independent Expert Anchors Matter: Aligning latent states to an independent, high-resolution medical vision encoder (MedSigLIP) yields massive accuracy gains compared to self-referential alignment (using the MLLM's own native vision tokens). Spatial resolution is a critical bottleneck for latent visual grounding.
- The Power of Discardable Scaffolding: VITAL proves that interpretability and efficiency are not mutually exclusive. By treating explanation generation as a training constraint rather than an operational requirement, models can remain lean at runtime while retaining auditability.
- Coarse-to-Fine Convergence: Qualitative analysis reveals that over successive latent steps ($z_1 \rightarrow z_4$), the visual projector's attention heatmaps naturally sharpen from broad anatomical regions down to tight, precise focuses on specific pathologies.
Key Results
- Massive Speedups over Explicit CoT: VITAL achieves a ~97× inference speedup compared to explicit CoT models (353ms vs. 34.1s) while delivering substantially higher accuracy.
- State-of-the-Art Performance: VITAL achieved an absolute accuracy improvement over its base model (Qwen3-VL-8B) of +47.52% on MSD and +46.67% on BiomedParse.
- Superior Visual Grounding: On the GEMeX-RMCoT visual grounding benchmark, VITAL scored 86.67%, outperforming trillion-parameter proprietary models like Claude-Opus-4.6 by an absolute margin of 28.34%.
- Strong Generalization: On the held-out, private in-house neuroimaging testset, VITAL achieved 80.50% accuracy, comfortably beating GPT-5.4 (73.80%).
Limitations and Future Work
- Fixed Reasoning Depth: VITAL currently enforces a fixed maximum of $K=4$ latent steps for every question. The authors note that implementing an adaptive early-exit mechanism (allowing the model to stop "thinking" early for simple questions) is a vital next step.
- Dependence on Teacher Quality: The textual reasoning chains used during training are distilled from a proprietary MLLM teacher, meaning VITAL's reasoning ceiling is fundamentally bounded by the teacher's medical accuracy.
- Conciseness vs. Verbosity Metrics: Because latent reasoning prioritizes concise correctness, VITAL scores lower on token-overlap metrics (like Token-F1) in open-ended generation setups that inherently reward verbose text outputs.
Summary updated