Can Image Generators Hold a Scene Together When Space Is Broken Apart?

Jhen-Ke Lin, Chung-Chun WangarXiv 2026 · arXiv

TL;DR

Current image generators are masters of local texture but struggle with global spatial reasoning. The authors introduce GJBench, a benchmark that asks models to generate scattered, rotated jigsaw pieces of a single scene, revealing that even state-of-the-art models fail to maintain scene continuity across non-adjacent regions of the canvas.

What Problem Does This Paper Address?

When an AI generates an image of a bridge, the two halves of the bridge are usually adjacent on the canvas. Because "belonging to the same object" and "being next to each other" are almost always synonymous in training data and benchmarks, it’s unclear if models truly understand a scene as a global entity or if they are just very good at making each pixel look consistent with its immediate neighbors.

The authors propose to disentangle these two properties by asking models to draw a scattered jigsaw puzzle. In this task, pieces of a single scene are moved apart and rotated on a plain background. To succeed, the model must "hold the scene together" in its internal representation even though the pieces are spatially disconnected on the output canvas.

Figure 1. Every system draws the picture, but none scatters it as a jigsaw. Five generators receive the same brief (a forested island in a crater lake). Top: each system’s picture drawn whole, which a program then cuts into nine interlocking pieces. Bottom: the same system asked for the nine pieces scattered on a plain background, from text alone. Labels give the pieces found and the interlock score (Sec. 4). Each exact cut above has 9 pieces and scores .94. The scene is chosen by a fixed rule.

Methods

The authors developed GJBench, which evaluates models on three distinct requirements:

  1. Scatter (RQ1): Does the image contain the requested number of separate pieces ($N=4, 9, 16$)?
  2. Interlock (RQ2): Do the shapes of the pieces actually fit together? The authors use a mixed-integer program (the HiGHS solver) to find the best possible physical arrangement of the generated pieces and score the fit using a Dice overlap metric.
  3. Reconstruction (RQ3): Does the content of the pieces, when reassembled, match a reference scene? This is measured using Multi-Scale Structural Similarity (MS-SSIM) and panoptic quality metrics.

The benchmark tests models under four conditions:

  • TEXT: Only a text prompt describing the scene.
  • REF: Text prompt plus a reference image of the whole scene.
  • LAYOUT: Text prompt plus a target layout showing piece outlines.
  • REF+LAYOUT: Both a reference image and a target layout.

Crucially, the evaluation is deterministic and pixel-based, avoiding the biases and "blind spots" often found in LLM-based or VLM-based judges.

Datasets

  • GJBench: A new benchmark consisting of 32 curated scenes from Wikimedia Commons (CC0). The scenes are divided into four categories to test different spatial properties: lines and topology (roads/rivers), object parts, multi-object relations (counting/relative positions), and scene geometry/lighting.
  • Models Evaluated: FLUX 3 Image Klein, Qwen-Image-2.1, GPT Image 2.6, Gemini Nano Banana 2.1, and Muse Image 1.0.

Key Insights and Ideas

  • Local vs. Global Coherence: Every model evaluated could draw the scenes perfectly as a "whole" image. However, every model failed to maintain that coherence once the scene was broken into scattered pieces.
  • Layout Helps Shape, Not Content: Providing a target layout (the outlines of the pieces) significantly improved the Interlock score, suggesting models can copy shapes. However, it did not improve Reconstruction, meaning models still couldn't place the right part of the scene into the right piece.
  • The Reference Image Paradox: Surprisingly, providing a reference image of the whole scene did not help models count pieces more accurately or make them fit together better. In some cases, it even made performance worse.
  • Failure Modes: Models failed in distinct ways. Some (like Qwen) tended to merge pieces together, while others (like Nano Banana) generated extra pieces. A common "cheat" was "thumbnailing"—drawing the entire scene in miniature inside each individual puzzle piece.
Figure 2. One output read by the three questions. (a) The pieces found on the background. (b) The best arrangement, with tab and socket pixels marked by whether the neighboring piece fills them. (c) Each piece on the reference cell it is paired with, labeled by its similarity to that cell. The output comes from GPT Image 2.6 with a reference and a layout, chosen by a fixed rule.

Key Results

  • GPT Image 2.6 was the most reliable at following the "scatter" and "interlock" instructions, but its reconstruction of scene content was still poor.
  • FLUX 3 struggled the most with interlocking; its pieces rarely had matching tabs and sockets unless a layout was explicitly provided.
  • Quantitative Gap: While the "control" (exact cuts of real images) scored ~0.94 on interlock and reconstruction, the best models typically scored below 0.50 on reconstruction.
  • Content Placement is Hardest: Even when models were given square tiles (no complex tabs) and told exactly where they belonged, they still couldn't accurately map the reference scene onto those tiles.

Limitations and Future Work

  • Proprietary "Black Boxes": Some models (like FLUX 3) use prompt upsamplers that rewrite user instructions. This makes it difficult to determine if the failure lies in the language understanding stage or the image generation stage.
  • Scale: The benchmark currently uses 32 scenes. Expanding this to more diverse content families would provide more robust statistics.
  • Future Directions: The authors suggest developing "reference-free" measures of spatial continuity, such as checking if the edges of scattered pieces are "mutual best matches" for each other, similar to how jigsaw puzzle solvers operate.

Summary updated

Abstract

Image generators are good at drawing scenes in which things that belong together appear next to each other. In almost every image they generate, and in almost every benchmark used to evaluate these images, "belonging to the same whole" and "adjacent to each other" go together. Therefore, we cannot know whether the model truly grasps the scene as a whole, or simply makes each part connect with its adjacent parts. We propose GJBench to disentangle the two. It requires the model to draw a scene as a scattered jigsaw puzzle: an image is cut into interlocking pieces, then moved apart and rotated. Thus, the spatial plausibility of the scene exists only in how these pieces collectively point to a whole. The model may have a reference image, a target layout, both, or neither. We separately evaluate three requirements: whether the scene is broken down into a specified number of separate pieces, whether the pieces interlock, and whether the pieces can reconstruct the reference scene. While every open and closed model we evaluated could draw these scenes well as a whole, none could keep them together as scattered puzzle pieces, and the three requirements categorized failures into different patterns. The models draw separate pieces but miscount them in different ways, and unless their outlines are provided, the pieces rarely interlock, and their content does not reconstruct the reference image. We also found that the target layout helps the puzzle pieces fit together, while the reference image improves neither their count nor their fit. These findings show that keeping a scene continuous across space that the canvas no longer shows remains an open problem for image generators.
BibTeX
@misc{lin2026can,
  author = {Jhen-Ke Lin and Chung-Chun Wang},
  title = {{Can Image Generators Hold a Scene Together When Space Is Broken Apart?}},
  year = {2026},
  eprint = {2611.20906},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.20906},
}
Figure