Slow Perception: Let's Perceive Geometric Figures Step-by-step

Haoran Wei, Youyang Yin, Yumeng Li, Jia Wang, Liang Zhao, Jianjian Sun, Zheng Ge, Xiangyu ZhangarXiv 2024 · DOI · arXiv

Abstract

While Large Vision Language Models (LVLMs) have made significant strides in general scene understanding, they continue to struggle with the precise geometric perception required for technical tasks. This paper introduces Slow Perception (SP), a framework designed to improve geometric figure parsing by forcing models to decompose complex shapes and trace lines incrementally. By employing a "perceptual ruler," the model generates a sequence of intermediate gaze points, revealing an inference-time scaling law where increased token generation leads to higher spatial accuracy. Our experiments on the SP-1 dataset and Mathvista benchmark demonstrate that "slowing down" perception significantly boosts both parsing precision and downstream mathematical reasoning.

Introduction: The Perception Gap in LVLMs

The recent emergence of reasoning-heavy models, such as OpenAI’s o1, has highlighted the potential for complex problem-solving in AI. However, the authors argue that a "visual o1" cannot exist without a foundation of robust perception. Currently, state-of-the-art models like GPT-4o and Claude 3.5 often fail at basic geometric figure parsing—the process of converting a 2D image into a structured, coordinate-based format.

The primary failure mode is the "long visual jump." When a model attempts to predict a line segment by identifying only its two endpoints in a single step, it often loses track of spatial relationships. This leads to misaligned vertices, disconnected shapes, and a general lack of topological consistency. To solve this, the authors propose that models should "read" an image with the same care a human student uses when tracing a figure with a ruler.

Methodology: Decomposing and Flowing

The Slow Perception (SP) framework introduces two levels of slowing down the model's output:

1. Perception Decomposition

Instead of treating a complex geometric figure as an indivisible object, the model first breaks it down into a unified list of basic units, such as lines and circles. This decomposition ensures that shared vertices—for instance, the vertex $B$ shared by segments $AB$ and $BC$—are referenced consistently throughout the output. This structural constraint prevents the "chaotic connections" common in standard LVLM outputs.

2. Perception Flow and the Perceptual Ruler

The core innovation of this work is Perception Flow. Inspired by human saccades and the act of tracing, the authors introduce a "perceptual ruler" of length $d$. Rather than predicting a line simply as (start, end), the model is trained to predict a sequence of intermediate "gaze points": start -> gaze point 1 -> gaze point 2 -> ... -> end.

The Perception Flow concept: tracing a line carefully using intermediate gaze points rather than a single long jump.

The number of steps $n$ is a function of the line's length relative to the ruler length $d$. By decreasing $d$, the model is forced to use more tokens to "trace" the line. This process increases the computational effort per line but drastically improves the model's grounding in the image's spatial layout.

Model Architecture

The architecture follows a standard LVLM pipeline: a VisionEncoder processes the image, a linear layer maps visual features to the text space, and a TextDecoder generates the coordinate sequence. The "slow" aspect is entirely contained within the target output format, which requires the decoder to output the full sequence of gaze points for every line.

The Slow Perception architecture and the resulting coordinate sequence for lines and circles.

Data and Evaluation

To support this research, the authors developed the SP-1 dataset, consisting of:

  • Synthetic Data (200,000 images): Rendered via Matplotlib, these images include various line thicknesses, styles (solid vs. dashed), and resolutions to ensure model robustness.
  • Real-world Benchmark (480 images): Manually annotated geometric figures from middle school math exams, providing a high-quality test for practical application.
  • Downstream Reasoning: A subset of 208 images from Mathvista was used to evaluate how improved perception impacts mathematical problem-solving.

Key Insights: The Inference Time Scaling Law

A major finding of this study is the Inference Time Scaling Law for Perception. Much like how Chain of Thought (CoT) allows LLMs to perform better by giving them "time to think," Slow Perception allows LVLMs to perform better by giving them "time to perceive."

As the perceptual ruler length $d$ decreases—meaning the model generates more tokens per unit of length—the accuracy of the final endpoints steadily improves. Interestingly, the authors found that the exact precision of the intermediate gaze points is less important than the process of sequential generation. Adding noise (jitter) to these points during training resulted in a negligible performance drop (less than 1%), suggesting that the sequential "flow" itself provides the necessary spatial stabilization.

Results and Impact

The Slow Perception method yielded significant improvements across multiple metrics:

  • Parsing Accuracy: On the SP-1 test set, moving from a baseline (infinite ruler) to a ruler of length 4 improved F1-scores from 51.4% to 57.5% (at 0.75 IoU).
  • Reasoning Gains: The most compelling result came from providing SP-generated coordinate "sketches" to GPT-4o on the Mathvista Geo-subset. The model’s reasoning accuracy jumped from 53.37% to 60.10%. This suggests that many failures previously attributed to "poor reasoning" are actually rooted in "poor perception."

Conclusion and Future Directions

Slow Perception demonstrates that the "cursory glance" of current LVLMs is a bottleneck for technical visual tasks. By slowing down the perception process, models can achieve human-like precision in geometric parsing.

However, this precision comes at a computational cost due to increased token generation. Future research will explore Reinforcement Learning to develop a "variable-length perceptual ruler." Such a system would allow the model to dynamically adjust its speed—skipping quickly over simple, clear segments and slowing down only for complex intersections or occluded regions, optimizing both efficiency and accuracy.

Summary updated

Abstract

Recently, "visual o1" began to enter people's vision, with expectations that this slow-thinking design can solve visual reasoning tasks, especially geometric math problems. However, the reality is that current LVLMs (Large Vision Language Models) can hardly even accurately copy a geometric figure, let alone truly understand the complex inherent logic and spatial relationships within geometric shapes. We believe accurate copying (strong perception) is the first step to visual o1. Accordingly, we introduce the concept of "slow perception" (SP), which guides the model to gradually perceive basic point-line combinations, as our humans, reconstruct complex geometric structures progressively. There are two-fold stages in SP: a) perception decomposition. Perception is not instantaneous. In this stage, complex geometric figures are broken down into basic simple units to unify geometry representation. b) perception flow, which acknowledges that accurately tracing a line is not an easy task. This stage aims to avoid "long visual jumps" in regressing line segments by using a proposed "perceptual ruler" to trace each line stroke-by-stroke. Surprisingly, such a human-like perception manner enjoys an inference time scaling law -- the slower, the better. Researchers strive to speed up the model's perception in the past, but we slow it down again, allowing the model to read the image step-by-step and carefully.
BibTeX
@misc{wei2024slow,
  author = {Haoran Wei and Youyang Yin and Yumeng Li and Jia Wang and Liang Zhao and Jianjian Sun and Zheng Ge and Xiangyu Zhang},
  title = {{Slow Perception: Let's Perceive Geometric Figures Step-by-step}},
  year = {2024},
  eprint = {2412.20631},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2412.20631},
}
Figure