TL;DR
Praxis-VLM introduces a novel training paradigm that teaches Vision Language Models (VLMs) complex situational decision-making using text-only reinforcement learning. By decoupling reasoning from visual perception during training via the GRPO algorithm, the model learns to "think before acting" using text descriptions—a capability that successfully transfers to multimodal visual tasks at inference time without requiring expensive paired image-text reasoning data.
What Problem Does This Paper Address?
While modern VLMs excel at visual perception, they often lack the multi-step situational reasoning required for complex decision-making in robotics, autonomous driving, or interactive assistants. Training models to reason typically requires large-scale, high-quality datasets pairing visual inputs with explicit textual reasoning chains—data that is notoriously expensive and difficult to curate.
The authors investigate a fundamental question: Is direct multimodal experience strictly necessary to learn these reasoning skills? They discover that foundational decision-making logic can instead be effectively acquired purely through language, bypassing the need for massive paired image-text reasoning datasets.
Methods
The authors propose Praxis-VLM, which utilizes a multi-stage Group Relative Policy Optimization (GRPO) framework paired with an adaptive reward mechanism:
- Stage 1: Foundational Reasoning Cold Start: The model (initialized with Qwen2.5-VL) is trained on logical and mathematical data (
geometry3k) to enforce a strict<think>...</think><answer>...</answer>format using rule-based rewards ($R_{tag}$, $R_{format}$, $R_{accuracy}$). This successfully bypasses the traditional, costly Supervised Fine-Tuning (SFT) phase typically used before reinforcement learning. - Stage 2: Text-Based Decision-Making RL: The model is trained on synthetic text-only situational descriptions to explore diverse reasoning paths. An adaptive reward ($R_{accuracy} + 0.8 \cdot R_{format} + 0.5 \cdot R_{len}$) encourages longer, more deliberate reasoning chains.
- Multimodal Inference: During test time, the entire VLM architecture—including its vision encoder—is reactivated. The textually learned reasoning skills seamlessly transfer to process actual visual inputs (images or videos) alongside textual questions.
Datasets
- Geometry3k: Used in Stage 1 for foundational logical structuring.
- Synthetic Text-Based Decision-Making Dataset: A newly created dataset consisting of 10K training and 1K validation samples generated via GPT-4o, where visual scenes are replaced by rich text descriptions paired with multiple-choice questions.
- Evaluation Benchmarks:
- VIVA: Human-centered situations evaluating social contexts and human values.
- PCA-Bench: Embodied agent tasks spanning robotics, autonomous driving, and games.
- EgoNormia: An out-of-domain egocentric video dataset requiring sequential and temporal reasoning.
Key Insights and Ideas
- Reasoning Disentanglement: The core "aha moment" is that fundamental decision-making logic can be completely disentangled from visual perception. Preliminary tests showed that standard VLMs perform comparably or even better when given text descriptions of a scene rather than the image itself.
- Zero-Shot Multimodal Transfer: Reasoning skills acquired purely from text descriptions successfully ground themselves in real visual data during inference without explicit cross-modal alignment training.
- Bypassing SFT: Unlike DeepSeek-R1 style pipelines that rely on an initial SFT phase to teach formatting, Praxis-VLM shows that an instruction-tuned VLM can learn formatting directly via GRPO using strict structural rewards ($R_{tag}$).
Key Results
- Praxis-VLM consistently outperformed both vanilla Qwen2.5-VL and SFT baselines across all benchmarks at both 3B and 7B scales.
- On the out-of-domain EgoNormia video benchmark, Praxis-VLM-7B achieved an accuracy of 54.33%, significantly outperforming the SFT baseline (34.83%), demonstrating robust generalization rather than pattern-mimicking behavioral cloning.
- Analysis of reasoning length showed that the model naturally generates longer reasoning chains for inherently more difficult samples, rather than producing redundant tokens.
Limitations and Future Work
- Sequence Truncation & Overthinking: Extremely long reasoning chains occasionally exceeded the maximum token limit (1,024 tokens) or introduced cognitive noise ("overthinking"), leading to drops in final decision accuracy.
- Perception-Reasoning Mismatch: Errors still occur due to misinterpreting complex visual contexts (e.g., missing a red light), safety prioritization failures, or misaligning with nuanced social norms.
- Future Directions: The authors suggest exploring advanced data selection strategies for text scenarios and explicitly improving the VLM's core visual grounding capabilities to better align visual features with textually learned abstract reasoning.
Summary updated