Vision-language models, systems that read an image and answer in text, describe a few isolated objects well. In crowded scenes they break down. They miscount, lose track of which object is which, and attach one object's color or size to its neighbor. Code-generating models in computer-aided design (CAD) have the opposite profile. They can turn a single clean part into an exact, editable program, a short Python script in a CAD library such as CadQuery that rebuilds the shape when run. They have never been asked to do this for a cluttered pile of parts. This note proposes building that missing system and argues that a program has one property that makes clutter tractable. A program can be executed and rendered, and the renderer knows exactly which object produced each pixel. That makes the program a self-checking hypothesis. Every object it claims can be compared, pixel by pixel, against the evidence in the image. Pixels that no object explains point to objects the system missed.
The core idea fits in two sentences. A model writes a scene program that places and shapes every object it believes is present, renders that program into depth and per-object masks, and receives a numeric error report broken down by object, plus a list of unexplained image regions. A second model then edits the program by refining, adding, splitting or deleting objects until the rendering agrees with the image. The claim is about scaling. Existing approaches commit to one reading of the image up front, and their accuracy falls steeply as object count and occlusion grow. Checking the scene program by executing it should make that fall much shallower, because miscounts and misattributed attributes appear as measurable disagreement instead of staying hidden inside the encoder.
Where this sits
Single-part reverse engineering into code is now strong. CAD-Recode maps point clouds to CadQuery scripts with a small language-model decoder trained on a million procedural programs. cadrille extends this to multi-view images and text, and adds online reinforcement learning that rewards the overlap (IoU, intersection over union) of the executed shape with the target and penalizes code that fails to compile. Both are evaluated only on isolated parts. That is the gap this proposal targets, and the method reuses their reconstructor and their execution-reward recipe. cadrille also reports that mixing synthetic and human-authored programs during supervised training hurts accuracy. That result constrains us: we train on a single program style.
Programs for multi-object scenes exist, but only in the forward direction. The Scene Language writes scenes as programs with loops, which gives 100% counting accuracy on prompts like "15 coke cans in a pyramid", against about 11% for 3D diffusion baselines. That result motivates using programs as the representation for counts. It works from text, though, and never has to parse a crowded image. SceneCraft generates Blender code for up to 100 assets and repairs it with a vision-language critic that looks at renders. Our design rejects that critic. The density literature shows such critics are exactly what fails under clutter: VisualOverload reports o3 at 19.6% on its hardest dense split and peak counting accuracy of 41.7%, and ChartMuseum reports drops of up to 30% as element density rises. SPADA gives the constructive version of the lesson. Code-repair agents fix CAD assemblies far more reliably from deterministic numeric logs ("part A intersects part B by 2mm") than from visual inspection of renders, which doubled its assembly pass rate from 20.4% to 41.9%. SPADA, however, needs a text specification to write its tests. Our contribution is a way to produce SPADA-style numeric feedback when the only specification is an image. The render-versus-image comparison serves as the test suite.
On the perception side, VISER shows that attribute scrambling in crowded images, the "binding problem", is reduced by overlaying horizontal grid lines and scanning row by row. It also shows that verbal chain-of-thought cannot repair an already scrambled visual encoding. Its grid is content-blind, cuts through objects, and degrades beyond about 50 items. We replace the grid with something that adapts to content: the current program's own object masks and unexplained regions decide where the system looks next. CObL splits occluded tabletop scenes into ordered, amodally completed image layers and reaches 83.5% top-1 adjusted Rand index (ARI) on TABLETOP. Its output is pixels with no geometry, so we treat it as both a component and a competitor.
Core Claim
Programs already serve as output formats. The new claim is that executing a candidate scene program gives a per-object error signal that a vision encoder alone cannot provide. Used inside an edit loop, that signal reduces the rate at which accuracy degrades with object count and occlusion, compared with segment-first pipelines, scaffolded vision-language models, and one-shot program generation given the same compute.
Concretely, let the image be $I$ and the current program $P$, which places objects $o_1,\dots,o_K$. Rendering $P$ gives a depth map $D_P$ and visible masks $M_k$, one per object. For each object we compute a residual
$$r_k = \frac{1}{|M_k|}\sum_{x \in M_k} \left|D_P(x) - \hat D(x)\right| + \lambda\,\mathrm{edge}_k,$$
where $\hat D$ is the observed depth (sensor depth when available, otherwise a monocular estimate such as the MiDaS depth CObL uses), $\mathrm{edge}_k$ is the mismatch between rendered silhouette edges and image edges along object $k$'s boundary, and $\lambda$ is a weighting constant. We also compute an unexplained set $U$: pixels that look like foreground in the image but that no rendered object covers. The editor receives these as a numeric log in the style of SPADA. It also receives image crops centered on the worst objects and on connected components of $U$, so that the single-part reconstructor sees one object at a time even in a crowded scene.
Research Program
The system has three parts. The first is an initial scene coder: a vision-language model fine-tuned to emit a CadQuery scene program from the image, with each object as a placed sketch-extrude part and loops wherever there is repetition. The second is a deterministic renderer that produces depth, normals and per-object masks. The third is an editor model that reads the residual log and crops and emits program edits: refine object $k$ (the crop goes through a cadrille-style single-part reconstructor), add an object in region $U_j$, split $o_k$ into two, delete $o_k$, or change a loop count. We train the editor with execution-reward reinforcement learning in the manner of cadrille. During training the reward is the change in matched scene IoU against ground truth. At test time the loop stops when residuals stop decreasing, so no ground truth is used at inference.
Training and controlled evaluation need a density-scaling generator. We sample parts from the DeepCAD and CAD-Recode procedural corpora and scatter them on a support surface with physics settling. We render from realistic viewpoints at controlled object counts (1, 2, 4, 8, 16, 32 and 64) and controlled occlusion fractions. Every scene comes with its ground-truth program. This produces the experiment the literature lacks: accuracy as a function of density for program de-rendering, with occlusion separated from object count.
Baselines
The naive reference point is a single pass of a fine-tuned vision-language model that emits the whole scene program from the image, with no execution feedback. It shares the backbone and training data of our initial coder and represents cadrille's recipe applied directly to scenes. Without any loop, we expect it to show the steep density curve that VisualOverload documents.
The strongest existing competitor is the segment-then-reconstruct pipeline. CObL or Mask2Former splits the image into objects, then cadrille reconstructs each object crop independently and the results are placed using depth. It is strong because it already individuates objects before coding them, which is the main thing our method claims to add. To keep the comparison fair, it gets the same reconstructor weights, the same number of reconstructor calls (best-of-$k$ sampling per object, scored by the same per-object residual), and the same depth input. On real tabletop scenes, the number to beat for visible segmentation is CObL's 83.5% top-1 ARI on TABLETOP, against 66.3% for Mask2Former, reported in CObL. We compute ARI from our rendered program masks. For per-object CAD accuracy, no absolute IoU for cadrille appears in the file. Its reported gains are relative: 3-9% IoU from reinforcement learning and an invalidity rate below 0.2%. We therefore re-run cadrille ourselves on isolated parts as a ceiling. No reported result was found on Lacuna for multi-object CAD scene de-rendering, so the synthetic benchmark numbers will all be new.
The scaffolding baseline is VISER's grid overlay with sequential scanning, applied to the same vision-language model before it emits the scene program. On counting, the number to beat is the VISER result on its 2D synthetic benchmark: Qwen2.5-VL rises from 5.8% to 40.8% and GPT-4o from 12.0% to 38.8%, as reported in VISER. We reimplement that generator from its description and answer counting questions by reading counts off the de-rendered program.
Three ablations isolate the key ingredient. In the first, the numeric per-object residual is replaced with a SceneCraft-style vision-language critique of side-by-side render and image, at an equal call budget. In the second, object attribution is removed: the editor sees only a global residual and the unexplained set, with no per-object breakdown. In the third, foveated crops are removed. Every method gets the same total budget of model calls and generated tokens. We beat the strongest baseline if our matched-object IoU and count accuracy match it at low density and exceed it at 16 or more objects or at 50% or higher occlusion, with a shallower fitted slope of accuracy against log object count.
Why not just…?
Why not just segment first and reconstruct each object? This is the real threat, and at low occlusion it may tie with our method. Its weakness is structural. A wrong segmentation, such as two touching parts merged, one part split at an occluder, or a heavily hidden part missed, is never revisited, because each crop is reconstructed on its own and nothing checks the whole scene. Under occlusion, a hidden object often explains residual only once its neighbors are modeled jointly. The occlusion sweep settles the question. If segment-then-reconstruct, given residual-scored best-of-$k$, matches us at high occlusion, then joint execution adds nothing beyond better per-object sampling, and we would report that result.
Why not just add VISER's grid and let the model write code directly? VISER's own reported limits suggest it falls short: lines that cut objects in two and degradation beyond about 50 items. Its own finding that text reasoning cannot fix scrambled encodings points the same way. A grid also gives no signal after generation. The 64-object condition and the counting comparison test this directly.
Why not just train end-to-end with reinforcement learning on scene IoU and sample many times? This is cadrille's recipe scaled to scenes, and it is the naive baseline given best-of-$N$ at equal compute. If sampling breadth matches our loop's iterative depth, then per-object attribution is unnecessary. The "global residual only" ablation separates the two explanations.
Why not just use higher resolution or a bigger model? VisualOverload reports that 4K input does not remove the failure under clutter, and that some model families get worse on dense tasks as they grow. We include a larger backbone at the single-pass setting as a check, not as a main line.
Proposed Experiments
The decisive experiment is the density curve on the synthetic generator: matched-object IoU (with objects matched to ground truth one-to-one by the Hungarian algorithm), count error, scene Chamfer distance and invalid-program rate, plotted against object count separately for low and high occlusion, for every method at equal budget. The core claim holds if our curve's slope is clearly shallower than the slopes of both segment-then-reconstruct and VISER-scaffolded generation above eight objects. The vision-critic ablation should reproduce SPADA's finding in an image-only setting, with numeric residuals beating visual critique, and the gap should widen with density. Two transfer checks follow. On TABLETOP, ARI computed from program masks is compared against CObL's 83.5%. On the reimplemented VISER counting benchmark, accuracy is compared against 40.8%. A final diagnostic counts how often edits of each type (add, split, delete, refine) fire as density rises. If add and split dominate at high density, that supports the claim that the loop's advantage comes from fixing individuation errors, which is the clutter-specific failure, rather than from polishing shapes.
References
- CAD-Recode: Reverse Engineering CAD Code from Point Clouds
- cadrille: Multi-modal CAD Reconstruction with Reinforcement Learning
- The Scene Language: Representing Scenes with Programs, Words, and Embeddings
- SceneCraft: An LLM Agent for Synthesizing 3D Scene as Blender Code
- VisualOverload: Probing Visual Understanding of VLMs in Really Dense Scenes
- ChartMuseum: Testing Visual Reasoning Capabilities of Large Vision-Language Models
- SPADA: A Verifiable Test-Driven Agent for Controllable Parametric CAD Assembly Generation
- Visual Structures Help Visual Reasoning: Addressing the Binding Problem in LVLMs
- CObL: Toward Zero-Shot Ordinal Layering without User Prompting
- DeepCAD: A Deep Generative Network for Computer-Aided Design Models