Shape of Thought: Progressive Object Assembly via Visual Chain-of-Thought
Lists
Only you see this
Abstract
Multimodal models for text-to-image generation have achieved strong visual fidelity, yet they remain brittle under compositional structural constraints—notably generative numeracy, attribute binding, and part-level relations. To address these challenges, we propose **Shape-of-Thought (SoT)**, a visual CoT framework for *process-supervised progressive shape assembly in the rendered 2D domain*, without external engines at inference time. SoT trains a unified multimodal autoregressive model to generate interleaved textual plans and rendered intermediate states, helping the model capture shape-assembly logic without producing explicit geometric representations. Unlike text-only CoT, each decision is grounded in a rendered state, making counts, attachments, topology, and intermediate part-addition errors inspectable across the trajectory. To support this paradigm, we introduce **SoT-26K**, a large-scale dataset of grounded assembly traces derived from part-based CAD hierarchies, and **T2S-CompBench**, a benchmark for evaluating structural integrity and trace faithfulness. Fine-tuning on SoT-26K achieves 88.4\% on component numeracy and 84.8\% on structural topology, outperforming direct generation by +24.2 points on component numeracy and +19.3 points on structural topology. SoT establishes a transparent testbed for rendered-domain structure-aware generation. The code is available at https://github.com/yuhuo03/Shape-of-Thought.
BibTeX
@inproceedings{huo2026shape,
author = {Yu Huo and Siyu Zhang and Kun Zeng and Haoyue Liu and Owen Lee and Junlin chen and YUQUAN LU and Yifu Guo and Yaodong Liang and Xiaoying Tang},
title = {{Shape of Thought: Progressive Object Assembly via Visual Chain-of-Thought}},
booktitle = {International Conference on Machine Learning},
year = {2026},
url = {https://openreview.net/forum?id=ZsJnkEgh6I},
}Loading related content...