When Words Outperform Vision: VLMs Can Self-Improve Via Text-Only Training For Human-Centered Decision Making

Zhe Hu, Jing Li, Yu Yin

Abstract

Embodied decision-making is fundamental for AI agents operating in real-world environments. While Visual Language Models (VLMs) have advanced this capability, they still struggle with complex decisions, particularly in human-centered situations that require deep reasoning about human needs and values. In this study, we systematically evaluate open-sourced VLMs on multimodal human-centered decision-making tasks. We find that LLMs receiving only textual descriptions unexpectedly outperform their VLM counterparts of similar scale that process actual images, suggesting that visual alignment may hinder VLM abilities. To address this challenge, we propose a novel text-only training approach with synthesized textual data. This method strengthens VLMs' language components and transfers the learned abilities to multimodal inference, eliminating the need for expensive image-text paired data. Furthermore, we show that VLMs can achieve substantial performance gains through self-improvement, using training data generated by their LLM counterparts rather than relying on larger teacher models like GPT-4. Our findings establish a more efficient and scalable approach to enhancing VLMs' human-centered decision-making capabilities, opening new avenues for optimizing VLMs through self-improvement mechanisms.

Introduction

While modern Vision Language Models (VLMs) excel at identifying objects within a scene, they often falter when faced with the "common sense" reasoning required for complex, human-centered decision-making. For instance, an autonomous agent might correctly identify a child near a stove but fail to prioritize immediate safety over routine tasks. This gap suggests that the bottleneck in current AI is not perception, but rather the high-level reasoning required to interpret visual data in the context of human values and social norms.

Traditionally, improving these models requires massive datasets of images paired with human-annotated reasoning chains—a process that is both expensive and difficult to scale. Praxis-VLM challenges this paradigm by asking: Is the core of decision-making ability tied to visual experience, or can it be learned purely through language? The researchers hypothesize that reasoning is a high-level skill that can be "pre-loaded" via text and subsequently grounded in vision during inference.

The Praxis-VLM Framework

Praxis-VLM introduces a multi-stage Reinforcement Learning (RL) approach designed to instill robust reasoning patterns without relying on image-text pairs during the primary training phase. The framework utilizes Group Relative Policy Optimization (GRPO), the same algorithm behind recent reasoning breakthroughs like DeepSeek-R1.

Multi-Stage Training

  1. Cold Start (Logical Structuring): The model is initially trained on geometry and mathematics problems. This phase establishes a logical "monologue" structure, where the model learns to use <think>...</think> tags to decompose problems before providing a final answer.
  2. Decision RL (Situational Reasoning): The model is then trained on 10,000 synthetic decision-making scenarios. These scenarios, generated via GPT-4o, cover diverse topics including workplace ethics, emergency handling, and resource allocation.
  3. Adaptive Reward System: To ensure depth of thought, a dynamic reward system scores the model based on format adherence, accuracy, and the length/complexity of its reasoning chain. This prevents the model from providing shallow or "cheated" answers.
Figure 1: An example of the Praxis-VLM reasoning process in a domestic robotics scenario. The model uses a <think> block to evaluate options before selecting the logical next step.

Experimental Setup and Benchmarks

The effectiveness of Praxis-VLM was evaluated across several rigorous benchmarks:

  • VIVA: A dataset focused on human-centered social situations and adherence to social norms.
  • PCA-Bench: A benchmark covering embodied AI tasks, including robotics and autonomous driving.
  • EgoNormia: A challenging out-of-domain benchmark involving egocentric (first-person) video understanding, testing the model's ability to generalize to new visual perspectives.

Key Results and Insights

The results demonstrate that text-only training significantly enhances multimodal performance. Praxis-VLM consistently outperformed both its base models and versions trained using standard Supervised Fine-Tuning (SFT).

  • Performance Gains: On the VIVA benchmark, Praxis-VLM-7B achieved 84.03% accuracy, compared to the base Qwen2.5-VL's 80.97%. On PCA-Bench, the improvement was even more dramatic, reaching 60.25% against the base model's 46.37%.
  • Robust Generalization: On the zero-shot EgoNormia dataset, Praxis-VLM maintained 54.33% accuracy. In contrast, the SFT-trained version plummeted to approximately 34%, suggesting that RL-based reasoning is far more robust to domain shifts than simple behavioral cloning.
  • The "Text Superiority" Phenomenon: Preliminary findings indicated that VLMs often perform better when provided with a text description of a scene rather than the image itself. By training on text, Praxis-VLM disentangles logic from perception, allowing the model to learn principles of safety and norms without the distraction of visual noise.
  • Four Dimensions of Thought: Analysis reveals that the model's reasoning naturally clusters into four critical areas: Situational Analysis, Action/Consequence Evaluation, Rule/Norm Adherence, and Safety/Risk Management.

Conclusion

Praxis-VLM proves that the reasoning capabilities of VLMs can be substantially improved through text-only self-improvement. By leveraging synthetic textual data and reinforcement learning, the model develops sophisticated "thinking" patterns that transfer seamlessly to visual tasks. This approach offers a scalable and efficient alternative to traditional multimodal training, paving the way for more capable and safer human-centered AI agents.

Summary updated

Abstract

Embodied decision-making is fundamental for AI agents operating in real-world environments. While Visual Language Models (VLMs) have advanced this capability, they still struggle with complex decisions, particularly in human-centered situations that require deep reasoning about human needs and values. In this study, we systematically evaluate open-sourced VLMs on multimodal human-centered decision-making tasks. We find that LLMs receiving only textual descriptions unexpectedly outperform their VLM counterparts of similar scale that process actual images, suggesting that visual alignment may hinder VLM abilities. To address this challenge, we propose a novel text-only training approach with synthesized textual data. This method strengthens VLMs' language components and transfers the learned abilities to multimodal inference, eliminating the need for expensive image-text paired data. Furthermore, we show that VLMs can achieve substantial performance gains through self-improvement, using training data generated by their LLM counterparts rather than relying on larger teacher models like GPT-4. Our findings establish a more efficient and scalable approach to enhancing VLMs' human-centered decision-making capabilities, opening new avenues for optimizing VLMs through self-improvement mechanisms.
BibTeX
@inproceedings{hu2025when,
  author = {Zhe Hu and Jing Li and Yu Yin},
  title = {{When Words Outperform Vision: VLMs Can Self-Improve Via Text-Only Training For Human-Centered Decision Making}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {2025},
  url = {https://openreview.net/forum?id=U806q3iILo},
}
Figure