Mind Meets Space: Rethinking Agentic Spatial Intelligence from a Neuroscience-inspired Perspective

Bui Duc Manh, Soumyaratna Debnath, Zetong Zhang, Shriram Damodaran, Arvind Kumar, Yueyi Zhang, Lu Mi, Erik Cambria, Lin WangarXiv 2025 · arXiv · OpenReview

Abstract

Recent advances in agentic AI have led to systems capable of autonomous task execution and language-based reasoning, yet their spatial reasoning abilities remain limited and underexplored, largely constrained to symbolic and sequential processing. In contrast, human spatial intelligence, rooted in integrated multisensory perception, spatial memory, and cognitive maps, enables flexible, context-aware decision-making in unstructured environments. Therefore, bridging this gap is critical for advancing Agentic Spatial Intelligence toward better interaction with the physical 3D world. To this end, we first start from scrutinizing the spatial neural models as studied in computational neuroscience, and accordingly introduce a novel computational framework grounded in neuroscience principles. This framework maps core biological functions to six essential computation modules: bio-inspired multimodal sensing, multi-sensory integration, egocentric-allocentric conversion, an artificial cognitive map, spatial memory, and spatial reasoning. Together, these modules form a perspective landscape for agentic spatial reasoning capability across both virtual and physical environments. On top, we conduct a framework-guided analysis of recent methods, evaluating their relevance to each module and identifying critical gaps that hinder the development of more neuroscience-grounded spatial reasoning modules. We further examine emerging benchmarks and datasets and explore potential application domains ranging from virtual to embodied systems, such as robotics. Finally, we outline potential research directions, emphasizing the promising roadmap that can generalize spatial reasoning across dynamic or unstructured environments.

Introduction

While Large Language Models (LLMs) and Vision-Language Models (VLMs) have mastered sequential reasoning and basic image description, they remain "spatially illiterate." Current AI agents often lack a genuine internal representation of the environment, leading to "hallucinations" when asked to navigate complex spaces or reason about object relationships (e.g., "what is behind that chair?").

The core issue is that contemporary AI is overly vision-centric and symbolic. In contrast, human spatial intelligence is rooted in multisensory integration (touch, sound, motion) and cognitive maps that allow us to switch seamlessly between our own perspective (egocentric) and a world-centered view (allocentric). This paper provides a roadmap for building agents that can truly "understand" the 3D space they inhabit by mimicking these biological structures.

A Neuroscience-Inspired Framework

To bridge the gap between artificial and biological intelligence, we propose a framework consisting of six essential modules that map human cognitive processes to agentic capabilities.

Mapping Human Spatial Intelligence to the Agentic Framework

1. Bio-inspired Multimodal Sensing

Effective spatial intelligence requires moving beyond cameras to include auditory, tactile, and proprioceptive (motion) sensors. This mimics the way humans perceive the world through multiple concurrent streams of data.

2. Multi-Sensory Integration

This module acts as the brain's information processing hub, calibrating, denoising, and selectively attending to the most relevant sensory inputs to create a coherent perception of the environment.

3. Egocentric-Allocentric Conversion

This system handles 3D scene abstraction, allowing an agent to transform what it sees from its own perspective (egocentric) into a stable, viewpoint-independent 3D map (allocentric). This is critical for maintaining spatial orientation during movement.

4. Artificial Cognitive Map

Implementing "Grid Cell" layers for metric navigation (distance/direction) and "Place Cell" layers for topological understanding (relationships between locations). As shown below, these maps evolve from simple Euclidean metrics to complex hybrid-hierarchical structures.

Evolution of Cognitive Map Representations from Metric to Hybrid-Hierarchical

5. Spatial Neural Memory

A dual-tier system featuring Episodic Memory (recalling specific past events) and Semantic Knowledge (general rules about how space works, such as "kitchens usually contain fridges").

6. Spatial Reasoning

Using "Predictive World Modeling" to simulate potential future actions and "Chain-of-Thought" reasoning to plan multi-step spatial tasks. This module allows the agent to minimize prediction error through active inference.

Benchmarking and Datasets

We categorize existing datasets by the cognitive process they evaluate:

  • 3D Perceptual Inference: NYU Depth-v2 (depth), ObjectNet3D (recognition), and newer multimodal sets like Ego-Exo4D (audio-visual) and 9DTact (tactile).
  • Hidden-State Inference: CLEVR (topological VQA), 3D Scene Graph, and MindCube (mental simulation).
  • Policy Selection: AI2-THOR, Habitat 2.0, and iGibson 2.0, which provide interactive 3D environments for testing autonomous agents.

Research Gaps and Future Directions

Our analysis identifies several critical Research Gaps (RGs) that must be addressed to achieve human-level spatial intelligence:

  1. Unified Sensing: Current sensing is modality-specific and lacks a unified, efficient fusion framework.
  2. Fragmented Scene Construction: Models either excel at geometry or semantics, but rarely both simultaneously.
  3. Brittle Neural Models: Deep learning models of grid/place cells often fail to generalize across different environments.
  4. Memory Persistence: Systems struggle with "catastrophic forgetting" when moving to new environments.
  5. Static Reasoning: Reasoning approaches are often too implicit (black box) or tied to specific, static inputs.

Future work should explore Neuromorphic Computing (using Spiking Neural Networks for event-driven sensing), Edge AI Deployment for local robotics processing, and Affective Reasoning to integrate social norms into spatial planning.

Conclusion

The path to advanced Agentic Spatial Intelligence lies in moving away from purely data-driven, symbolic models toward architectures that respect the structural and functional principles of the mammalian brain. By integrating multimodal sensing with persistent cognitive maps, we can create agents capable of navigating and reasoning within the complex, dynamic 3D world.

Summary updated

Abstract

Recent advances in agentic AI have led to systems capable of autonomous task execution and language-based reasoning, yet their spatial reasoning abilities remain limited and underexplored, largely constrained to symbolic and sequential processing. In contrast, human spatial intelligence, rooted in integrated multisensory perception, spatial memory, and cognitive maps, enables flexible, context-aware decision-making in unstructured environments. Therefore, bridging this gap is critical for advancing Agentic Spatial Intelligence toward better interaction with the physical 3D world. To this end, we first start from scrutinizing the spatial neural models as studied in computational neuroscience, and accordingly introduce a novel computational framework grounded in neuroscience principles. This framework maps core biological functions to six essential computation modules: bio-inspired multimodal sensing, multi-sensory integration, egocentric-allocentric conversion, an artificial cognitive map, spatial memory, and spatial reasoning. Together, these modules form a perspective landscape for agentic spatial reasoning capability across both virtual and physical environments. On top, we conduct a framework-guided analysis of recent methods, evaluating their relevance to each module and identifying critical gaps that hinder the development of more neuroscience-grounded spatial reasoning modules. We further examine emerging benchmarks and datasets and explore potential application domains ranging from virtual to embodied systems, such as robotics. Finally, we outline potential research directions, emphasizing the promising roadmap that can generalize spatial reasoning across dynamic or unstructured environments. We hope this work will benefit the research community with a neuroscience-grounded perspective and a structured pathway. Our project page can be found at Github.
BibTeX
@misc{manh2025mind,
  author = {Bui Duc Manh and Soumyaratna Debnath and Zetong Zhang and Shriram Damodaran and Arvind Kumar and Yueyi Zhang and Lu Mi and Erik Cambria and Lin Wang},
  title = {{Mind Meets Space: Rethinking Agentic Spatial Intelligence from a Neuroscience-inspired Perspective}},
  year = {2025},
  eprint = {2509.09154},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2509.09154},
}
Figure