Spatial Reasoning via Discrete Coordinate Tokenization

You are reading version 1 of this direction, frozen 2026-10-04. The latest version is version 2.

Version 1 of 2187 papers223 conceptsfrozen 2026-10-04

In traditional computer vision, locating an object is a regression task: a model predicts four continuous numbers representing a bounding box. In natural language processing, however, models predict discrete tokens from a fixed vocabulary. For years, these two worlds were bridged by "detect-then-reason" pipelines, where a frozen object detector passed cropped regions to a language model.

A new direction in multimodal research, exemplified by frameworks like OFA and UniTAB, collapses this distinction. By quantizing continuous spatial coordinates into discrete "location tokens," researchers are treating geometry as just another language. This allows a single Transformer to read and write positions as easily as it reads and writes words.

The Grid as a Vocabulary

The core mechanism is straightforward: an image is partitioned into a fixed grid of bins (e.g., a $1000 \times 1000$ grid). Each bin index is added to the model's vocabulary as a specialized token. To describe a bounding box, the model doesn't output raw pixel coordinates; it generates a sequence of four tokens—$[x_{min}, y_{min}, x_{max}, y_{max}]$—representing the corners of the box.

As Zhengyuan Yang and colleagues demonstrate in UniTAB, this discretization allows for "inline" grounding. A model can generate a caption like "A [cat] <obj> [bin_42, bin_88, bin_150, bin_200] </obj> is sitting on a mat," weaving spatial data directly into the natural language sequence. This unification eliminates the need for task-specific detection heads or complex geometric loss functions.

Why Discretization Works

It might seem counterintuitive to turn precise continuous numbers into "fuzzy" discrete bins. However, this shift offers several structural advantages:

  1. Unified Optimization: When coordinates are tokens, the model can be trained using standard cross-entropy loss. Chaoyang Zhu and the team behind SeqTR found that this classification-based approach is often more stable and data-efficient than coordinate regression, allowing smaller models to outperform much larger architectures.
  2. Tokenizer Efficiency: Using dedicated integer tokens is significantly more efficient than representing coordinates as strings of digits (e.g., "0.425"). Kanchana Ranasinghe and Satya Narayan Shukla show in LocVLM that standard LLM tokenizers struggle with decimal sequences, whereas discrete bins leverage the model's existing ability to handle categorical data, reducing sequence length and hallucination.
  3. Structural Scaffolding: In generative tasks, discrete tokens provide a "hard" constraint that natural language lacks. Chenfei Wu and Zhengyuan Yang argue in ReCo that using position tokens provides a more reliable structural scaffold for image generation than fuzzy positional words like "on the left," which models often ignore.

Scaling to 3D and Geospatial Domains

The intuition behind 2D image grids is now being extended to more complex dimensions. In LLaMA-Mesh, Z. Maggie Wang applies this logic to 3D space, quantizing vertex coordinates into discrete bins to allow a standard LLaMA model to generate 3D meshes as if they were text.

Similarly, in the geospatial domain, MoRA uses a hierarchical hexagonal grid to discretize the Earth's surface. This allows models to process human mobility and satellite data as sequences of spatial tokens, transforming global navigation into a structured data reasoning task. As Jieren Deng shows with IMAIA, this grid-based reasoning is significantly faster and more accurate for identifying points of interest than processing raw map imagery.

From Perception to Action

Perhaps the most significant shift is moving from passive perception to active agency. Models like Magma and Rex-Omni use discrete spatial tokens to bridge the gap between "seeing" and "doing." By quantizing 2D screen coordinates and 3D robotic joint states into the same vocabulary, Lars Liden and others enable LLMs to perform action planning. In this paradigm, clicking a button on a UI or moving a robotic arm is simply the result of the model predicting the next "spatial token" in a sequence.

By treating the physical world as a discrete vocabulary, these models move beyond mere description, gaining the ability to reason about, navigate, and manipulate the spaces they perceive.

Papers in this version

Showing 60 of 187 papers.

History & citationVersion 2 · 2026-10-04

Current · Permanent link

  1. Version 2 (latest) · 2026-10-04 · 105 papers
  2. Version 1 · 2026-10-04 · 187 papers
Cite this direction

Latest version (a living page):

@misc{lacuna_discrete_spatial_tokenization_9ad4ecda,
  title = {discrete spatial tokenization},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/discrete-spatial-tokenization/rdr_9ad4ecdafc5e410387b724571499537f},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-21},
}

This exact version (version 1, frozen text):

@misc{lacuna_spatial_reasoning_via_discrete_coordinat_9ad4ecda_v1,
  title = {Spatial Reasoning via Discrete Coordinate Tokenization},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/spatial-reasoning-via-discrete-coordinate-tokenization/rdr_9ad4ecdafc5e410387b724571499537f/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-21},
}