Abstract
VisuoThink is a novel "slow thinking" framework designed to enhance Large Vision-Language Models (LVLMs) by integrating step-by-step tool use for visual aid generation with a predictive tree search algorithm. By interleaving visual and textual reasoning and exploring multiple inference-time paths, VisuoThink enables models to solve complex geometry and spatial reasoning tasks dynamically. Experimental results demonstrate state-of-the-art performance across multiple frontier models without the need for fine-tuning.
Introduction
Recent advancements in Large Language Models (LLMs), such as OpenAI’s o1, have highlighted the power of "slow thinking"—scaling compute at inference time to allow for deliberate, step-by-step reasoning. While effective for text-based tasks, extending this paradigm to multimodal domains like geometry and spatial navigation has remained challenging. Current LVLMs often treat images as static inputs, lacking the dynamic, iterative visual-verbal reasoning humans use when sketching auxiliary lines or updating maps to solve problems. VisuoThink addresses this gap by introducing a multimodal test-time search strategy that interleaves visual feedback with textual planning.
Methodology
VisuoThink operates through two primary mechanisms that enable multimodal test-time scaling:
1. Vision-Text Interleaved Thinking
The model functions within a continuous Thought-Action-Observation loop. It plans a visual hint, utilizes an external tool (such as Python/matplotlib) to generate or update an image, and then incorporates the new visual feedback into its next reasoning step. This allows for dynamic problem-solving, such as drawing auxiliary lines in geometry or updating agent positions in navigation tasks.
2. Predictive Rollout Search
Inspired by Monte Carlo Tree Search (MCTS), VisuoThink explores multiple potential reasoning paths rather than relying on a single greedy output. At each reasoning step, the framework performs three actions:
- Expansion: Generates multiple candidate visual-textual actions.
- Rollout Simulation: Simulates the remainder of the reasoning path for each candidate to predict the final outcome.
- Selection: Employs a self-voting mechanism to choose the most promising path based on the simulated results.
Experimental Evaluation
The framework was evaluated on complex geometry and spatial reasoning benchmarks:
- Geometry: Evaluated on Geometry3K and Geomverse-109 (a high-quality, Python-executable subset of Geomverse).
- Spatial Reasoning: Tasks included Visual Navigation (2D grid navigation with obstacles) and Visual Tiling (polyomino composition), requiring explicit, executable action sequences.
Key Results and Insights
VisuoThink achieved significant performance gains across frontier models like GPT-4o, Claude-3.5-Sonnet, and Qwen2-VL-72B without any fine-tuning.
- Performance Gains: On Geomverse-109, VisuoThink outperformed standard Chain-of-Thought (CoT) by an average of ~17%. In spatial reasoning, the gains were even more pronounced; on Level-3 Visual Navigation, GPT-4o reached 93.8% accuracy compared to 18.8% for CoT.
- The "Goldilocks" Tree Width: Analysis revealed an inverted U-shape trend regarding tree width. While increasing candidate paths initially helps, an excessively wide tree can overwhelm the model's evaluation capabilities, leading to performance degradation.
- Supervision Quality: The effectiveness of predictive rollouts is highly dependent on environmental feedback. "Strong" visual supervision (e.g., map updates) yields massive boosts, while "weak" supervision (e.g., final numerical answers in geometry) provides more modest improvements.
Conclusion
VisuoThink demonstrates that multimodal inference-time scaling can significantly enhance the reasoning capabilities of LVLMs. While the framework introduces computational overhead and relies on external tool execution, it provides a robust path toward more human-like, dynamic visual reasoning. Future work will focus on optimizing search efficiency and expanding the framework to broader real-world applications.
Summary updated