TL;DR
The authors introduce MindView, a rigorous benchmark for generative multi-view spatial intelligence that challenges models to generate consistent new views of a scene based on partial observations and language queries. Their findings reveal a surprising "context scaling degradation"—where models perform worse when given more images—and propose MindCue, a training-free evidence selector that fixes this by intelligently picking the most relevant visual cues.
What Problem Does This Paper Address?
While we’ve seen massive leaps in AI's ability to answer spatial questions (e.g., "Is the chair to the left of the table?"), evaluating its ability to generate spatially consistent views remains a frontier. Most existing benchmarks focus on simple image editing or single-view synthesis.
The core challenge addressed here is generative multi-view spatial intelligence: given a handful of images of a room and a command like "move 30° left around the cabinet," can a model accurately predict and render what that new viewpoint looks like? This requires the model to integrate evidence across multiple views, infer the target camera's position, and maintain strict geometric and semantic consistency with the observed scene.
Methods
The researchers developed a comprehensive evaluation framework consisting of:
- The MindView Benchmark: A set of 1,500 samples across seven tasks categorized into two families:
- Geometry-Dominant: Explicit camera motions (Translation, Rotation, and their Composition).
- Semantics-Dominant: Viewpoints inferred from object references and spatial relations (e.g., "Relational Object Grounding" or "Multi-Cue Viewpoint Inference").
- BAGEL-SFT: To see if task-specific training helps, they fine-tuned the BAGEL model on a newly curated dataset of 20,000 training examples.
- MindCue: A novel, training-free spatial evidence selector. Instead of overwhelming a model with every available context image, MindCue uses a "target-view hypothesis" to select only the most relevant views. It leverages VGGT for geometric reconstruction and SAM 3 for semantic segmentation to identify which context images actually "see" the area the target camera is pointed at.
Datasets
The benchmark is built upon three high-quality indoor scene datasets:
- ScanNet
- ScanNet++
- ARKitScenes
MindView provides 1,500 test samples from 513 scenes and a separate training set of 20,000 examples from 3,406 scenes. Unlike previous datasets, MindView specifically filters for "connected paths" of views to ensure models have enough context to infer the 3D structure of the scene.
Key Insights and Ideas
- The "More is Less" Paradox: In a counterintuitive finding, all eleven evaluated generators (including GPT-Image-2 and FLUX.2) performed worse when the context was expanded from six views to ten. Even though the ten views provided better coverage of the scene, the models struggled to filter out irrelevant information, leading to degraded performance.
- Generation Aids Understanding: Fine-tuning a model to generate views (BAGEL-SFT) significantly improved its performance on three separate spatial understanding (QA) benchmarks, even though it wasn't trained on those benchmarks' specific annotations. This suggests that learning the "how" of spatial rendering naturally improves the "what" of spatial reasoning.
- Composition is Hard: Models consistently struggled most with "Composition" tasks (e.g., "turn left, then move right") and "Multi-Cue" tasks, highlighting a weakness in multi-step spatial reasoning and complex viewpoint inference.
Key Results
- Benchmark Performance: Closed-source models like GPT-Image-2 currently lead the pack with an overall score of 57.9, but there is still massive room for improvement compared to ground truth.
- Fine-tuning Gains: BAGEL-SFT achieved a massive 24.3-point gain over the base BAGEL model, outperforming even the best closed-source models in geometric consistency (RScore and TScore).
- MindCue Success: By selecting only relevant evidence, MindCue reversed the performance drop seen with expanded context. On BAGEL-SFT, it improved the overall score by 1.9 points over the 10-view baseline and outperformed random view selection.
Limitations and Future Work
The authors acknowledge that MindView currently uses a single real target image per query. For some semantic tasks (like "Relational Object Grounding"), there might be multiple valid viewpoints that satisfy the description. While they used expert review and clustering to minimize ambiguity, future iterations could benefit from multi-target evaluation.
Additionally, while MindCue helps with evidence selection, the underlying generators still struggle with fine-grained texture preservation and perfect geometric alignment. Future work could explore tighter integration between geometric solvers (like VGGT) and the generative diffusion process to ensure "hallucinated" content remains physically plausible.
Summary updated