Thinking Toward the Prototype: Design Fixation as Visual Hallucination in Multimodal Models

When a multimodal model (a system that reads images and text and answers in text) is shown an unusual chair, such as a cantilevered seat with no back legs, and is asked to reason about how to improve it, it may end up talking about the ordinary four-legged chair it expects rather than the one in the picture. This note proposes that two problems usually studied apart are one failure seen from two sides. Design researchers call it design fixation: the tendency to anchor on familiar, canonical solutions. Hallucination researchers call it ungrounded generation: fluent claims the input does not support. The specific claim is that longer step-by-step reasoning pulls a model's account of a design back toward the category prototype. The mechanism is the same attention drift away from image tokens already documented on math and perception benchmarks. This pull shows up both as false statements about the input artifact and as redesign proposals that converge on the stereotype.

If this holds, AI-side design fixation stops being a vague output-diversity complaint and becomes a measurable grounding failure with a known mechanism and a testable remedy. We would build a benchmark of atypical artifacts whose structure is known exactly. On it we would trace a fixation–grounding curve across reasoning budgets, and we would test whether bringing visual evidence back into the context partway through reasoning moves that curve outward. The alternative is that it merely trades novelty for accuracy, which is what simply reasoning less would do.

Core Claim

The motivating evidence comes from two literatures that do not cite each other. Understanding Design Fixation in Generative AI compared 96 GPT-4o/Midjourney chair designs with 105 Red Dot award winners. It reports a novelty ratio of 67.4% for ChatGPT versus 78.2% for humans, "buzzword" anchoring, texture-level rather than structural innovation, a near-universal 45-degree rendering angle, and novices adopting the model's first idea uncritically. That study treats the models as black boxes and uses separate text and image systems, so it cannot say why the fixation arises.

More Thinking, Less Seeing? supplies a candidate mechanism. As reasoning chains lengthen, attention shifts from image tokens to generated text, and the model reports what its language priors expect rather than what is shown. BabyVision reports that removing chain-of-thought raised attention to image tokens 5.28x and improved fine-grained visual accuracy. MIRAGE finds that spatial hallucinations such as left/right and front/behind barely shrink with scale. These papers study math diagrams and toddler-level visual tasks. None of them study artifacts whose typical form is a strong prior, which is exactly where the drift should be most damaging.

Our hypothesis joins the two. "What the model expects to see" for a designed object is its category prototype. Attention drift should therefore produce canonical intrusion, meaning assertions of prototype features absent from the image, and prototype pull, meaning redesigns closer to the category centroid than to the input. Both should grow with reasoning length. The hallucination framing also explains a puzzle in Chen et al.: models do surface-level texture changes on standard silhouettes because the structural silhouette they reason about is the prototype, not the input.

Research Program

The first component is a benchmark, tentatively called Atypical Artifacts. It covers six product categories with strong prototypes: chairs, lamps, kettles, bicycles, tables, and door handles. For each category we collect real award-winning or unconventional designs, seeded with the Red Dot chair set used by Chen et al. We pair these with procedurally edited 3D models whose structural primitives are known exactly. For example, a chair can be given three legs, a cantilever base, a sling seat, or no backrest. Each edit is rendered from several viewpoints, deliberately including non-45-degree views. This gives exact ground truth about which features are present.

Following the obscure-features idea from the design literature (the observation that people miss non-typical physical properties of familiar objects), every item carries a structural inventory. The inventory states support type, number of contact points, load path, joints, and openings, and it flags which entries deviate from the category prototype. Each item has two kinds of question. Perception questions ask directly about deviant features. Design questions ask the model to critique or modify the artifact while preserving a named property of the input, such as "keep the cantilever." This forces reasoning over the actual structure.

The second component is diagnosis. We vary reasoning budget in three ways: no chain-of-thought, standard chain-of-thought, and forced longer thinking budgets on reasoning models. At each budget we measure four things. Canonical-intrusion rate is the fraction of structural assertions that claim prototype features contradicted by the inventory, judged against ground truth rather than by a language model. Prototype pull is measured by embedding the model's described redesign, and its rendered version when we pass it to a fixed image generator, then computing similarity to the category centroid minus similarity to the input. Novelty uses the keyword-based ratio of Chen et al. for comparability. Image-attention mass is measured per reasoning step, as in Xu et al. We then test whether a step's image-attention mass predicts whether that step introduces a canonical intrusion, which is the mechanistic link the hypothesis needs.

The third component is an intervention called claim-triggered visual re-grounding. It has two parts. First, the model fills the structural inventory with short, direct answers and no chain-of-thought, each tied to an image crop. This exploits BabyVision's finding that direct answering keeps attention on pixels. Second, during free reasoning, whenever the model asserts a structural property of the input, the relevant crop is re-inserted into the context and the claim is checked against the inventory before reasoning continues. The point is to let the model deliberate at length, which design ideation needs, while repeatedly restoring image tokens near the end of the context, where drift has moved attention away. This targets the tension the file identifies between reflective prompting that helps text factuality (Reflection of Thoughts, 62.9% overall consistency on clinical guidelines) and verbal reasoning that harms visual grounding.

A small fourth component tests whether the effect reaches people. Novice designers do a 30-minute redesign session with an assistant that is either the base reasoning model or the re-grounded variant. We measure how often participants adopt hallucinated canonical features into their final concept. This tests the "double fixation" concern that fluent ungrounded rationales lock novices into prototype-bound ideas.

Baselines

The naive reference point is the same base model, Qwen2.5-VL-7B and InternVL from opengvlab/internvl, answering directly without chain-of-thought. It is the strongest grounding reference, because BabyVision and Xu et al. both predict it hallucinates least. It anchors one end of the fixation–grounding curve, and any method must beat it on design novelty without losing much grounding.

The second baseline is standard chain-of-thought plus forced longer thinking budgets, which traces the other end of the curve.

The strongest existing grounded-reasoning model is Ocean-R1-7B, the reinforcement-learning-only model that More Thinking, Less Seeing? reports at the top RH-AUC of 0.63 on RH-Bench. We also include the Logos method from MIRAGE, which reports MIRAGE accuracy rising from 28.8% to 37.1% for Logos-7B over Qwen2.5-VL-7B. If Logos weights are not available we will use its test-time hint component only and say so. M2-Reasoning serves as a spatial-reasoning-trained comparison.

Because our intervention re-inspects crops, the most direct competitors are "thinking with images" zoom/crop agents of the kind audited in Beyond Accuracy. That paper reports a 57% ungrounded-correct-answer rate for Thyme and 78% grounding for Mini-o3. We will also include training-free DCoT box prompting, which reports MM-Vet 38.4% versus 36.1% for LLaVA-1.5-13B. Finally, we include Reflection-of-Thoughts verbal deliberation.

All baselines run on the same images and questions. Prompted methods get the same number of prompt-tuning iterations on a held-out category. We cap generated tokens at matched budgets, counting re-inserted crops toward our method's cost. No reported result was found on Lacuna for any method on Atypical Artifacts, since the benchmark is new.

Beating the baselines means that, at matched total tokens, re-grounding lies strictly outside the convex hull of the baselines' (canonical-intrusion, novelty) points across budgets. Concretely, it must have lower intrusion than standard chain-of-thought at equal novelty, and higher novelty than no-chain-of-thought at equal intrusion, with confidence intervals from bootstrapping over items.

Three ablations isolate the key ingredient: inventory without mid-reasoning re-insertion, re-insertion of the whole image at fixed intervals rather than claim-triggered crops, and re-insertion of a random crop. The random-crop ablation tests whether the benefit comes from visual evidence or merely from refreshing image tokens.

Why not just…?

Why not just turn off chain-of-thought? This is the most serious threat. BabyVision shows it raises image attention and visual accuracy, so it may well minimize canonical intrusion on its own. Our bet is that it also suppresses the multi-step deliberation that structural redesign needs, giving low intrusion but low novelty. The no-chain-of-thought baseline and the fixation–grounding curve settle this directly. If no-chain-of-thought matches re-grounding on novelty, we will report that AI design fixation is cured by reasoning less and drop the intervention claim.

Why not just prompt for diversity or raise sampling temperature? Diversity tools like DiverXplorer and reference recombination in CreativeConnect do widen output spread. However, Chen et al.'s "surface-level innovation" finding suggests that diversity can coexist with a fixed prototype silhouette. We include a high-temperature, diversity-prompted condition. Our prediction is that it raises novelty while leaving canonical intrusion unchanged or worse. The specific signature is varied textures on a hallucinated four-legged base.

Why not just use a zoom/crop "thinking with images" agent? These agents already re-look at the image, so they may already solve this. The evidence in Beyond Accuracy is that they often crop the wrong region and answer from priors. That points to a gap between when they look and what they claim. Our method ties looking to structural claims. The head-to-head at matched crop counts decides whether that coupling matters, and a tie would mean existing agents suffice.

Why not caption first and reason in text? GeoSQA found reasoning failed even from perfect expert descriptions, and BabyVision argues language loses spatial structure. Still, our ground-truth inventory makes a strong oracle variant cheap to run: reason over the true inventory with no image. If this oracle text pipeline beats everything, the problem is perception rather than drift, and that is a useful negative result in its own right.

Decisive Checks

The diagnostic claim fails if canonical intrusion does not increase with reasoning budget on deviant features, or if per-step image-attention mass does not predict which steps introduce intrusions. The test is a held-out logistic model of intrusion given attention mass, compared against a step-position-only baseline. A further test is a control set of typical artifacts. On those, prototype intrusion is by definition correct, so if longer reasoning helps there while hurting on atypical items, that asymmetry is the fingerprint of prior-driven drift rather than generic degradation.

The intervention claim fails if it does not move the fixation–grounding frontier relative to no-chain-of-thought, or if the random-crop ablation matches it.

The human-study claim fails if novices' adoption of hallucinated features does not differ between assistants. Each failure is informative. Together they determine whether design fixation in multimodal models is best treated as a grounding problem, a diversity problem, or a perception problem.