SpatialMind: Adaptive Multi-Level Geometric Fusion for Spatial Reasoning in Multimodal Large Language Models

Shibin Zhao, Hao Tan, Fan Yang, Haixin Wang, Feilong Chen, Yidong Ma, Chaoyang Zhao, Ming Tang, Jinqiao WangarXiv 2026 · arXiv

TL;DR

SpatialMind is a task-adaptive framework that enables Multimodal Large Language Models (MLLMs) to perform superior 3D spatial reasoning by dynamically fusing geometric features at different granularities. By moving away from rigid, "one-size-fits-all" fusion strategies, it achieves state-of-the-art performance (72.3 on VSI-Bench) across diverse spatial intelligence benchmarks.

What Problem Does This Paper Address?

While MLLMs have become remarkably proficient at 2D visual tasks, they still struggle with 3D spatial reasoning—tasks like estimating distances, understanding room layouts, or planning navigation routes.

Existing attempts to fix this usually involve injecting 3D geometric priors (like point clouds or depth information) into the model. However, these methods typically use a fixed fusion scheme: they inject the same level of geometric detail regardless of whether the task is simple (e.g., "how big is this table?") or complex (e.g., "plan a path to the door").

The authors argue that spatial tasks are hierarchical. Low-level perception requires fine-grained, local details (shallow features), while high-level reasoning requires a holistic understanding of scene structure and object relationships (deep features). A fixed fusion strategy cannot optimally serve both.

Figure 1: SpatialMind: Adaptive Multi-Level Geometric Fusion for Spatial Reasoning. Unlike prior methods that rely on fixed geometric fusion despite the varying granularity requirements of different spatial tasks, SpatialMind adaptively selects task-appropriate features, from local details to global scene structures. (Red and green indicate higher and lower attention values, respectively) (b) Comparison of geometric fusion strategies, with results in (c) showing that SpatialMind achieves competitive performance across spatial intelligence benchmarks.

Methods

The core contribution is SpatialMind, which introduces a decoupled, task-adaptive fusion architecture.

  1. GeoFus Module: This is a multi-layer geometric fusion module that replaces fixed injection. It uses a gating network conditioned on three inputs: the textual instruction, visual semantics from the image, and the geometric representations themselves.
  2. Multi-Level Geometric Projection: The model extracts features from different layers of a geometry encoder (specifically VGGT). These represent a "progressive geometric hierarchy":
  3. Shallow features: Capture fine-grained details like edges and boundaries.
  4. Intermediate features: Capture object-level configurations.
  5. Deep features: Capture global scene layouts and inter-object relationships.
  6. Adaptive Gating: For every task, GeoFus calculates weights to mix these shallow, middle, and deep features dynamically before injecting them into the LLM decoder layers.
  7. Negative-Entropy Regularization: To prevent the model from "collapsing" (i.e., always picking just one layer and ignoring the others), the authors introduce a regularization loss that encourages the gating network to explore and utilize complementary information across levels during training.
The SpatialMind architecture featuring the GeoFus module for task-adaptive integration.

Datasets

The authors trained SpatialMind on a massive, balanced corpus of spatial reasoning data, including:

  • VG-LLM (SPAR and LLaVA-Hound subsets)
  • ScanNet (from VLM-3R)
  • VSI-590K (curated subset)
  • ViCA-322K, MindCube, and Spatial-MLLM-203K

For evaluation, they tested across six major benchmarks: VSI-Bench (primary), SPAR-Bench, SPBench, BLINK, MindCube, and SQA3D.

Key Insights and Ideas

  • Hierarchical Specialization is Real: Visualization of attention maps confirms that shallow layers focus on local contours (like sofa cushion boundaries), while deep layers focus on holistic scene understanding (like room-level furniture layout).
  • Task-Granularity Alignment: Quantitative ablations show that removing shallow features hurts low-level tasks (like distance estimation) the most, while removing deep features severely degrades high-level reasoning (like route planning).
  • Dynamic Fusion > Fixed Fusion: The "aha moment" is that the model doesn't just need more geometry; it needs the right geometry. By allowing the model to "choose" its granularity based on the prompt, it handles the cognitive spectrum of spatial tasks much more effectively.
Figure 3: Effects of Multi-Level Geometric Features. Shallow features contribute more to low-level tasks, while deep features are more important for high-level tasks, demonstrating hierarchical specialization and cross-level complementarity.

Key Results

  • State-of-the-Art Performance: SpatialMind-10B achieved a score of 72.3 on VSI-Bench, setting a new record for open-source spatial models and outperforming several proprietary APIs.
  • Consistency Across Scales: The 5B version also performed exceptionally well (71.5), proving that the adaptive fusion mechanism provides gains independent of the backbone's parameter count.
  • Broad Generalization: Beyond VSI-Bench, SpatialMind ranked first on five out of six benchmarks in the 3-5B category, showing robust performance across multi-view reasoning, 3D QA, and mental simulation tasks.
Figure 2: Visualization of Multi-Level Geometric Attention across 2D Views and 3D Point Clouds. Features at different lev- els capture spatial information at distinct scales, progressing from fine-grained details to global layouts.

Limitations and Future Work

  • Fine-grained Metric Perception: While strong, the model still has room for improvement in high-precision, single-view metric estimation (e.g., exact centimeter-level distance).
  • General Visual Logic: Because the training was heavily spatial-centric, the model's performance on non-spatial visual-logical tasks (like solving jigsaw puzzles or art style recognition in the BLINK benchmark) is secondary to its spatial prowess.
  • Future Directions: The authors suggest that incorporating more diverse visual supervision and stronger metric-geometry estimation modules could further close the gap in fine-grained perception.

Summary updated

Abstract

Multimodal Large Language Models (MLLMs) demonstrate promising capabilities in 2D visual understanding, yet remain limited in 3D spatial reasoning. Recent studies explore integrating 3D reconstruction models into MLLMs. However, these approaches either use a single geometric granularity across all tasks or adopt a fixed multi-level fusion scheme, regardless of task-specific reasoning requirements. Through systematic experiments on multi-level geometric fusion, we find that spatial tasks at different cognitive levels (e.g., distance estimation vs. route planning) require geometric representations of different granularities, revealing that fixed fusion cannot accommodate diverse spatial reasoning requirements. To bridge this gap, we propose SpatialMind, a task-adaptive framework for dynamically integrating multi-level geometric representations. SpatialMind employs GeoFus, a multi-layer geometric fusion module built around a gating network jointly conditioned on textual instructions, visual semantics, and geometric representations. This network adaptively modulates the contributions of shallow, intermediate, and deep geometric features across the layers of the LLM decoder. Furthermore, we introduce a negative-entropy regularization loss to mitigate gate collapse. Gate visualizations indicate that GeoFus adapts geometric granularity to task requirements. Experiments demonstrate that SpatialMind achieves competitive performance across spatial benchmarks, attaining a state-of-the-art score of 72.3 on VSI-Bench. The source code is publicly available at https://github.com/zhaosb-star/SpatialMind.
BibTeX
@misc{zhao2026spatialmind,
  author = {Shibin Zhao and Hao Tan and Fan Yang and Haixin Wang and Feilong Chen and Yidong Ma and Chaoyang Zhao and Ming Tang and Jinqiao Wang},
  title = {{SpatialMind: Adaptive Multi-Level Geometric Fusion for Spatial Reasoning in Multimodal Large Language Models}},
  year = {2026},
  eprint = {2611.22415},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.22415},
}
Figure