FutureMapping: The Computational Structure of Spatial AI Systems.

Andrew J. DavisonarXiv 2018 · OpenReview

Introduction

Simultaneous Localisation and Mapping (SLAM) is undergoing a fundamental transformation. As argued by Andrew Davison, SLAM is evolving into "Spatial AI"—a persistent, metric, and semantic understanding of 3D space that enables embodied agents to interact intelligently with their environment. However, the transition from bulky research prototypes to consumer-grade wearables like AR glasses or low-cost household robots faces a significant hurdle: the "power wall."

The Performance Gap and the Power Wall

Current state-of-the-art SLAM and deep learning algorithms are computationally expensive, typically requiring high-end GPUs and substantial power. For "holy grail" applications—such as AR glasses weighing under 70g that can operate all day—spatial perception must be achieved within a sub-1W power budget. Moore’s Law no longer provides the serial processing gains needed to bridge this gap. Instead, we must rethink the computational structure of how AI perceives the world, moving away from general-purpose CPUs toward specialized architectures that prioritize power efficiency and data locality.

Hardware-Algorithm Co-design

The "FutureMapping" roadmap proposes a shift toward Computational Graph architectures. Rather than treating memory as a flat, centralized resource, both the world model (the Map Graph) and the algorithms (the Computation Graph) should be mapped onto hardware that reflects their natural connectivity. This involves a deep co-design of hardware and algorithms.

Key to this approach is the use of specialized processors like Intelligence Processing Units (IPUs), designed for sparse graph operations, and in-plane sensors that perform low-level vision processing directly on the image sensor. This reduces the massive energy cost associated with data transmission between the sensor and a central processor.

The Spatial AI Brain: Data Locality and In-Plane Processing

Davison envisions a "Spatial AI Brain" where the "Map Store" is distributed across thousands of independent cores. In this model, data lives where it is processed, mirroring the "small world" topology of biological brains.

Architecture of a programmable vision chip featuring a 256x256 sensor/processor array for massively parallel low-level vision processing.

As shown in the figure above, programmable vision chips like the SCAMP5 utilize a sensor/processor array where each pixel or small group of pixels has dedicated analog and digital processing circuitry. This allows for extremely low-latency, low-power feature extraction and motion estimation before data ever leaves the sensor plane.

Semantic Evolution and Model-Based Sensing

Spatial AI is progressing through three levels: from sparse localization (Level 1) to dense geometric mapping (Level 2), and finally to semantic, object-level understanding (Level 3). A critical innovation in this journey is "model-based event sensing." While current event cameras report changes in pixel intensity, a model-based approach would only transmit data that disagrees with the system's internal 3D model, drastically reducing bandwidth and power consumption.

Conclusion

Achieving the 1W target for ubiquitous Spatial AI requires a fundamental shift in how we build perception systems. By embracing data locality, graph-based representations, and hardware-algorithm co-design, we can move beyond the limitations of current computing paradigms toward truly embodied intelligence. Future benchmarks must evolve to reflect these priorities, measuring not just accuracy, but also the energy cost of data movement.

Summary updated

Abstract

We discuss and predict the evolution of Simultaneous Localisation and Mapping (SLAM) into a general geometric and semantic `Spatial AI' perception capability for intelligent embodied devices. A big gap remains between the visual perception performance that devices such as augmented reality eyewear or comsumer robots will require and what is possible within the constraints imposed by real products. Co-design of algorithms, processors and sensors will be needed. We explore the computational structure of current and future Spatial AI algorithms and consider this within the landscape of ongoing hardware developments.
BibTeX
@misc{davison2018futuremapping,
  author = {Andrew J. Davison},
  title = {{FutureMapping: The Computational Structure of Spatial AI Systems.}},
  year = {2018},
  eprint = {1803.11288},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1803.11288},
}
Figure