Mind the Gap: Geometrically Accurate Generative Reconstruction from Disjoint Views

Grzegorz Wilczynski, Mikolaj Zielinski, Bartosz Swirta, Dominik Belter, Przemyslaw SpurekarXiv 2026 · DOI

TL;DR

This paper introduces a new paradigm for 3D reconstruction from "disjoint views" – images that don't overlap at all. Their GLADOS framework uses generative AI to "bridge the gap" between these views, creating a coherent 3D scene while ensuring the original images are accurately represented. This is crucial for real-world applications like robotics and casual 3D capture where overlapping views are often impossible.

What Problem Does This Paper Address?

Traditional 3D reconstruction methods, from classic Structure-from-Motion to modern Neural Radiance Fields (NeRFs) and 3D Gaussian Splatting, heavily rely on visual overlap between input images. This overlap is essential for establishing correspondences, aligning camera poses, and building a consistent 3D model. However, this requirement is a major bottleneck in many practical scenarios:

  • Robotics & Exploration: Swarms of drones or robots exploring an area will naturally capture disjoint views to maximize coverage, leading to massive spatial gaps in their data.
  • Casual Capture: Users taking photos of their homes for virtual tours often capture separate rooms without overlap, leaving gaps in the overall scene.
  • Crowdsourced Data: Large-scale 3D reconstructions from internet photos often have fragmented views due to popular viewpoints, missing connections between key areas.

Existing methods fail catastrophically in these zero-overlap scenarios, producing disconnected or semantically incoherent 3D models. Generative models can "hallucinate" missing content, but they often struggle to enforce strict geometric constraints from the original, disjoint observations. This paper tackles the challenge of creating a single, unified 3D environment from entirely non-overlapping images, treating the original views as hard geometric constraints while generating the intermediate space.

Methods

The authors propose a three-stage framework called GLADOS (Generative Reconstruction from Disjoint Views):

  1. Generative Spatial Bridging: This is the core innovation. It uses a Vision-Language Model (VLM) to analyze the input disjoint images and generate a descriptive text prompt. This prompt then guides a diffusion model to synthesize a plausible "anchor" image that semantically and geometrically bridges the gap between the original views. This synthesized anchor creates artificial overlap, allowing traditional reconstruction techniques to proceed.
  2. Robust Coarse 3D Reconstruction: With the synthesized anchor, the pipeline now has a set of (nearly) overlapping images. They use an unconstrained dense pointmap regression model (like Dust3r) to establish a coarse 3D Gaussian Splatting scaffold. This stage is designed to be robust to potential generative inconsistencies in the anchor image.
  3. Iterative Context Expansion and Consistency Optimization: This stage refines the coarse scaffold. It involves:
  4. Warp-and-Inpaint: Rendering the scene from novel viewpoints and using an RGB inpainting prior (guided by the initial text prompt) to fill missing visual context.
  5. Metric Depth Alignment: Using a monocular depth estimator and aligning its output to ensure structural integrity.
  6. Latent Consistency Model: A diffusion model is used to enforce multi-view consistency across the generated views, rectifying diverging geometries and textures.
  7. Generative 3D Refinement: A final iterative loop uses anchored grid inpainting and super-resolution to fill residual holes and improve textural fidelity, grounding the generation to the original ground-truth observations.

Datasets

The authors introduce a new disjoint-view benchmark for evaluating reconstruction in zero-overlap scenarios. This benchmark is built by sampling non-overlapping frame pairs from existing datasets:

  • Mip-NeRF 360
  • Tanks & Temples
  • Deep Blending
  • RealEstate10K

They also add 7 additional scenes collected specifically for this work, totaling 22 scenes. They also define specialized evaluation metrics (GeCo, MEt3R, Photo) tailored for this sparse-reference setting.

Key Insights and Ideas

  • Bridging the "Semantic & Spatial Gap": The core idea is that generative models can be used not just to create scenes, but to connect existing, disconnected ones by synthesizing plausible intermediate content.
  • Generative Priors as "Glue": The VLM-guided diffusion model acts as a powerful "glue" that can infer the missing spatial and semantic relationships between disjoint views.
  • Two-Stage Approach for Robustness: The pipeline cleverly combines the generative power of diffusion models with the geometric rigor of established reconstruction methods. The coarse reconstruction stage acts as a low-pass filter to smooth out generative artifacts before a more detailed refinement.
  • Grounding Generation to Reality: The final refinement stage is crucial for ensuring that the generated content is not just plausible but also geometrically accurate and consistent with the original observations.

Key Results

  • State-of-the-Art Performance: GLADOS significantly outperforms existing reconstruction (Dust3r, VGGT, AnySplat) and generative (VistaDream) methods on the proposed disjoint-view benchmark across various geometric and perceptual metrics (e.g., MEt3R, GeCoM, GeCoD, GeCoF, FID, CLIP Score).
  • No Reconstruction Failures: Unlike other methods that often fail to produce a valid 3D reconstruction in disjoint scenarios, GLADOS achieves a 0% failure rate across all tested scenes.
  • Generative Spatial Bridging Boosts Baselines: Augmenting existing methods with their Generative Spatial Bridging module generally improves their performance and reduces failures, demonstrating the utility of synthesized anchors.
  • Qualitative Coherence: Visual results show GLADOS producing visually coherent, plausible, and geometrically accurate reconstructions, effectively filling gaps without introducing obvious artifacts or distortions.

Limitations and Future Work

  • Unconstrained Regions: The authors acknowledge that in heavily unconstrained regions with insufficient observational constraints, the framework relies on generative priors, which may introduce semantically plausible but geometrically inaccurate structures.
  • Potential Misuse: The ability to create realistic spatial reconstructions from sparse observations could potentially be misused. The authors mitigate this by explicitly distinguishing between observed and generated content and treating generated regions as probabilistic scene completions.
  • Future Work: The paper suggests that the proposed framework has broad impact potential in robotics, autonomous exploration, virtual environment generation, and cultural heritage preservation. Further research could explore improving robustness in highly unconstrained scenarios and mitigating potential misuse.

Summary updated

BibTeX
@misc{wilczynski2026mind,
  author = {Grzegorz Wilczynski and Mikolaj Zielinski and Bartosz Swirta and Dominik Belter and Przemyslaw Spurek},
  title = {{Mind the Gap: Geometrically Accurate Generative Reconstruction from Disjoint Views}},
  year = {2026},
  eprint = {2605.07550},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2605.07550},
}
Figure