Praxis-VLM: Vision-Grounded Decision Making via Text-Driven Reinforcement Learning

Zhe Hu, Jing Li, Zhongzhu Pu, Hou Pong Chan, Yu Yin

TL;DR

Praxis-VLM introduces a novel training paradigm that teaches Vision Language Models (VLMs) complex situational decision-making using text-only reinforcement learning. By decoupling reasoning from visual perception during training via the GRPO algorithm, the model learns to "think before acting" using text descriptions—a capability that successfully transfers to multimodal visual tasks at inference time without requiring expensive paired image-text reasoning data.

What Problem Does This Paper Address?

While modern VLMs excel at visual perception, they often lack the multi-step situational reasoning required for complex decision-making in robotics, autonomous driving, or interactive assistants. Training models to reason typically requires large-scale, high-quality datasets pairing visual inputs with explicit textual reasoning chains—data that is notoriously expensive and difficult to curate.

The authors investigate a fundamental question: Is direct multimodal experience strictly necessary to learn these reasoning skills? They discover that foundational decision-making logic can instead be effectively acquired purely through language, bypassing the need for massive paired image-text reasoning datasets.

Methods

The authors propose Praxis-VLM, which utilizes a multi-stage Group Relative Policy Optimization (GRPO) framework paired with an adaptive reward mechanism:

  • Stage 1: Foundational Reasoning Cold Start: The model (initialized with Qwen2.5-VL) is trained on logical and mathematical data (geometry3k) to enforce a strict <think>...</think><answer>...</answer> format using rule-based rewards ($R_{tag}$, $R_{format}$, $R_{accuracy}$). This successfully bypasses the traditional, costly Supervised Fine-Tuning (SFT) phase typically used before reinforcement learning.
  • Stage 2: Text-Based Decision-Making RL: The model is trained on synthetic text-only situational descriptions to explore diverse reasoning paths. An adaptive reward ($R_{accuracy} + 0.8 \cdot R_{format} + 0.5 \cdot R_{len}$) encourages longer, more deliberate reasoning chains.
  • Multimodal Inference: During test time, the entire VLM architecture—including its vision encoder—is reactivated. The textually learned reasoning skills seamlessly transfer to process actual visual inputs (images or videos) alongside textual questions.
Figure 3: Overview of Praxis-VLM: Learning transferable reasoning from text-only data for mul- timodal decision-making. The process involves (1) constructing synthetic text-based training data where situations are represented through textual descriptions, (2) training the VLM on this data using RL with adaptive rewards to develop reasoning skills, and (3) transferring the learned reasoning to vision-grounded decision-making tasks during inference.

Datasets

  • Geometry3k: Used in Stage 1 for foundational logical structuring.
  • Synthetic Text-Based Decision-Making Dataset: A newly created dataset consisting of 10K training and 1K validation samples generated via GPT-4o, where visual scenes are replaced by rich text descriptions paired with multiple-choice questions.
  • Evaluation Benchmarks:
    • VIVA: Human-centered situations evaluating social contexts and human values.
  • PCA-Bench: Embodied agent tasks spanning robotics, autonomous driving, and games.
  • EgoNormia: An out-of-domain egocentric video dataset requiring sequential and temporal reasoning.

Key Insights and Ideas

  • Reasoning Disentanglement: The core "aha moment" is that fundamental decision-making logic can be completely disentangled from visual perception. Preliminary tests showed that standard VLMs perform comparably or even better when given text descriptions of a scene rather than the image itself.
Figure 2: Model accuracy on VIVA [18] and PCA-Bench [19]. Image Situation uses the original image as input, and Text Situa- tion employs the caption (text) instead.
  • Zero-Shot Multimodal Transfer: Reasoning skills acquired purely from text descriptions successfully ground themselves in real visual data during inference without explicit cross-modal alignment training.
  • Bypassing SFT: Unlike DeepSeek-R1 style pipelines that rely on an initial SFT phase to teach formatting, Praxis-VLM shows that an instruction-tuned VLM can learn formatting directly via GRPO using strict structural rewards ($R_{tag}$).

Key Results

  • Praxis-VLM consistently outperformed both vanilla Qwen2.5-VL and SFT baselines across all benchmarks at both 3B and 7B scales.
  • On the out-of-domain EgoNormia video benchmark, Praxis-VLM-7B achieved an accuracy of 54.33%, significantly outperforming the SFT baseline (34.83%), demonstrating robust generalization rather than pattern-mimicking behavioral cloning.
  • Analysis of reasoning length showed that the model naturally generates longer reasoning chains for inherently more difficult samples, rather than producing redundant tokens.
Accuracy vs reasoning length across quintiles

Limitations and Future Work

  • Sequence Truncation & Overthinking: Extremely long reasoning chains occasionally exceeded the maximum token limit (1,024 tokens) or introduced cognitive noise ("overthinking"), leading to drops in final decision accuracy.
  • Perception-Reasoning Mismatch: Errors still occur due to misinterpreting complex visual contexts (e.g., missing a red light), safety prioritization failures, or misaligning with nuanced social norms.
  • Future Directions: The authors suggest exploring advanced data selection strategies for text scenarios and explicitly improving the VLM's core visual grounding capabilities to better align visual features with textually learned abstract reasoning.

Summary updated

Abstract

Vision Language Models exhibit impressive performance for various tasks, yet they often lack the sophisticated situational reasoning required for complex decision-making. This paper shows that VLMs can achieve surprisingly strong decision-making performance when visual scenes are replaced by textual descriptions, suggesting foundational reasoning can be effectively learned from language. Motivated by this insight, we propose Praxis-VLM, a reasoning VLM for vision-grounded decision-making. Praxis-VLM employs the GRPO algorithm on textual scenarios to instill robust reasoning capabilities, where models learn to evaluate actions and their consequences. These reasoning skills, acquired purely from text, successfully transfer to multimodal inference with visual inputs, significantly reducing reliance on scarce paired image-text training data. Experiments across diverse decision-making benchmarks demonstrate that Praxis-VLM substantially outperforms standard supervised fine-tuning, exhibiting superior performance and generalizability. Further analysis confirms that our models engage in explicit and effective reasoning, underpinning their enhanced performance and adaptability.
BibTeX
@inproceedings{hu2025praxis,
  author = {Zhe Hu and Jing Li and Zhongzhu Pu and Hou Pong Chan and Yu Yin},
  title = {{Praxis-VLM: Vision-Grounded Decision Making via Text-Driven Reinforcement Learning}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {2025},
  url = {https://openreview.net/forum?id=U806q3iILo},
}
Figure