Visual Abstract Thinking Empowers Multimodal Reasoning

Dairu Liu, Ziyue Wang, Minyuan Ruan, Fuwen Luo, Chi Chen, Peng Li, Yang LiuarXiv 2025 · DOI · arXiv

Abstract

Multimodal Large Language Models (MLLMs) often struggle with visual reasoning due to the high density of redundant information in raw images. We present VAT (Thinking with Visual Abstract), a prompting paradigm that supplements original images with simplified visual abstracts, such as sketches or edge maps. VAT filters out visual noise, allowing models to focus on structural essentials. Our results show that VAT outperforms standard Chain-of-Thought (CoT) methods in both accuracy and efficiency across multiple benchmarks, including a new "Odd-One-Out" task.

The Challenge of Visual Redundancy

While images are rich in information, they are often too rich for efficient machine reasoning. Raw pixels contain immense amounts of redundant data—textures, lighting, and background clutter—that can distract a model from core tasks like counting objects or understanding spatial relationships.

Existing solutions generally rely on Verbal Chain-of-Thought (CoT), which is often verbose and expensive, or Visual Tools that use external agents to segment images, adding significant computational overhead. VAT seeks to replicate human cognitive strategies by mentally simplifying complex scenes into core concepts before reasoning.

Methodology: Thinking with Visual Abstracts

The VAT framework involves a two-step process:

  1. Visual Abstraction: Input images are transformed into "abstracts" using techniques ranging from simple code-based methods (Canny edge detection, Binarization) to deep-learning-based styles (PhotoSketch, OpenSketch).
  2. Dual-Image Prompting: The MLLM is prompted with both the original raw image and the visual abstract. The abstract serves as a "perceptual guide," helping the model focus on contours and structural features without requiring lengthy intermediate text.
Figure 1: Comparison of VAT Prompting against Standard and CoT methods on the Odd-One-Out task. VAT correctly identifies the target by focusing on abstract structural differences.

Key Insights and Results

The study evaluates VAT across a wide array of benchmarks, including BLINK, CoSpace, and a new Odd-One-Out benchmark where models must identify a single different object in a grid.

  • Performance Gains: VAT achieved an average gain of +2.21% over the GPT-5 baseline, significantly outperforming the +0.96% gain seen with standard CoT. The benefits were most pronounced in larger models; GPT-4o saw a 15.33% improvement over its CoT baseline in certain tasks.
  • Efficiency: Because VAT doesn't require the model to output hundreds of intermediate "thought" tokens, it is much faster than CoT or tool-based agents. On GPT-4o, VAT required only 0.28x the runtime of tool-using methods and consumed 90% fewer tokens.
  • Style Matters: Not all abstracts are equal. OpenSketch, which retains geometric depth and salient edges, generally performed best. Interestingly, combining multiple abstract styles often decreased performance by re-introducing redundancy.
Figure 2: Efficiency vs. Accuracy across different methods and tasks. VAT (red) consistently occupies the high-accuracy, high-efficiency (lower log budget) quadrant.

Conclusion

VAT demonstrates that "less is more" in visual reasoning. By providing a denoised structural guide, MLLMs can achieve higher accuracy with lower computational costs. While the abstraction process can sometimes strip away fine-grained textures needed for specific tasks, the overall paradigm shift from verbal to visual abstract thinking represents a significant step forward in multimodal model efficiency.

Summary updated

Abstract

Images usually convey richer detail than text, but often include redundant information, which potentially downgrades multimodal reasoning performance. When faced with lengthy or complex messages, humans tend to employ abstract thinking to convert them into simple and concise abstracts. Inspired by this cognitive strategy, we introduce a novel paradigm to elicit the ability to Think with Visual Abstract (VAT), by prompting Multimodal Large Language Models (MLLMs) with visual abstract instead of explicit verbal thoughts or elaborate guidance, permitting a more efficient visual reasoning mechanism via concentrated perception. VAT encourages models to focus on more essential visual elements, concepts and structural features by undermining redundant information compared with explicit thinking methods, such as Chain-of-thought (CoT) and tool-using approaches, that increase the complexity of reasoning process via inserting verbose intermediate steps and external knowledge. Experimental results show that VAT consistently empowers different MLLMs in visual perception and reasoning tasks. VAT achieves an average gain of $2.21\%$ over GPT-5 baseline, surpassing the gain of CoT, demonstrating that VAT better enhances multimodal task performance of MLLMs. Additionally, VAT spends fewer tokens while achieving higher performance. These findings highlight the effectiveness of visual abstract thinking and encourage further exploration of more diverse reasoning paradigms from the perspective of human cognition.
BibTeX
@misc{liu2025visual,
  author = {Dairu Liu and Ziyue Wang and Minyuan Ruan and Fuwen Luo and Chi Chen and Peng Li and Yang Liu},
  title = {{Visual Abstract Thinking Empowers Multimodal Reasoning}},
  year = {2025},
  eprint = {2505.20164},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2505.20164},
}
Figure